Import big data into AWS as fast as possible


Product name


Scroll down

A quick look

Our client needed to import big data as fast as possible but at the same time ensure enough cloud infrastructure capacity to run the client's e-commerce website and mobile app.

Implementation schema

Our Mission

Build a scaling mechanism to spin-up or kill-off Fargate ECS services processing Messaging Queues.

Our Approach

Our solution was a Lambda function which scales the ECS services processing the Messaging Queues. The Lambda function is based on an algorithm that calculates the available IOPS of the database, in order to make sure that the ECS services process the Messaging Queues as fast as possible, without causing any interruption in the availability of the web page.


  • Easy communication between the Lambda and ECS via the Python AWS SDK
  • The cloudwatch event to trigger the Lambda in a desired time period
  • Prioritization of the ECS tasks via a configuration file in the Lambda. This was impossible to achieve with a native ECS cron solution for example.
  • Accurate calculation of the available RDS IOPS

More of our work

AI notifications about content you like

Scalable universal platform for sending notification about topics chosen by people.

See more

Online used car sales platform

Online cross border used car sales platform that uses data from all European car sales platforms to create a pricing map and evaluates the most interesting and price competitive cars to offer.

See more