AWS athena

AWS Athena Error: Query exhausted resources at this scale factor

Athena is a Serverless technology i.e. It make use of shared resources available with AWS and hence, when large amount of queries are submitted by users concurrently around the world at the same time, sometimes resource exhaustion take place.  Athena service team has identified this as a known issue.

However, this error is transient in nature. By that I mean if you can submit the query again, it might be successful. If you repeatedly get the same error consistently, then you might need to partition your data and optimize the query further as mentioned in Performance Tuning Best Practices for Athena [1] and another aritcle in this blog about cost reduction which in turn might reduce resource consumption [2].

You can find suggestions below  from AWS support team:

1) Avoid submitting queries at the beginning or end of an hour. If query fails, Back off exponentially by some minutes and try to submit query again. [ Wierd, but thats an official answer…]
2)  highly recommended to adopt Amazon Athena best practices [1] to optimize your query and your data.
3) Use columnar formatted data which can drastically reduce the resource consumption.

[1] Top 10 Performance Tuning Best Practices for Athena —

[2]  aws atehna Cost reduction (might also reduce resource consumption)


I put a lot of thoughts into these blogs, so I could share the information in a clear and useful way. If you have any comments, thoughts, questions, or you need someone to consult with, feel free to contact me:

Leave a Reply