Request: customize CodeBuild container size

[Kimblis]

Hey, I asked this question in gitter, however, this seem more appropriate place since I can add more information (screen shots). Not sure if it is related specifically to copilot tho, however, I would really love your help.
I'm trying to run unit and e2e tests before every deployment. My architecture is microservices (have 11 federated microservices in general).
My build look like this:

It calls make file:

Which basically calls 3 scripts.
All 1st script is doing is clonin all microservices from github and preparing env file.
2nd script composes up mysql, mongo, redis and rabbit servers and does a docker-run to all microservices to install packages and run db migrations, and then composes these docker containers as well.
And my 3rd and the last script just calls test commands:

That's the error I get:

Honestly, not sure why this is happening, it's like process quits for whatever reason. (test config should be fine since it works wonders locally, right?)
Also, I tried to clone the same service alongside with other services, and run these commands there- same problem.
I can provide whatever additional info you need.
Any help/ideas are welcome

[huanjani]

Hello, @Kimblis!

Hmmm...what if you break out the commands in your third script-- what happens if you run npm run test:unit directly? Let's see what output you get then!

[Kimblis]

They're already splitted, it's two different commands (npm run test:unit and npm run test:e2e) :/

[huanjani]

Oh, sorry-- I meant to run the commands directly in the buildspec, rather than within the script within run-tests.

[Kimblis]

Oh, sorry-- I meant to run the commands directly in the buildspec, rather than within the script within run-tests.

Just tried that. Didn't work :/ Same problem. Any other ideas?

[huanjani]

🤔 Just brainstorming still...are all of the dependencies in script 2 up to date? I've seen other cases of success locally but not in the build being fixed by upgrading. 🤷🏼 It may we worth a try while we keep thinking on this. 😄

[Kimblis]

Yeah, that did not help, however I dug deeper into these container status codes. It seems that my container "runs out of memory". Any idea how to fix that? How big is that linux container that we get for codebuild? It should def be enough to run some tests? (We do not have that many tests huh)? Maybe somehow my containers aren't using all provided memory or something like that? Any ideas?

[Kimblis]

Yeah, now I'm pretty sure that it's the memory issue. How big is my kernel? I think that it has some memory limitations for some actions and is not using all memory. Is it possible to increase memory for that main container (kernel I guess). For docker containers inside I could use docker update -m 10g container_name, but I guess I'm not running these tests from those containers (even if I want to). Is it possible to increase memory on that main container, or should I try to run these tests inside dockerized containers that I create inside? That's probably a bit confusing explanation, however I hope you understand where am I getting with this :D

[Lou1415926]

I wonder if increasing the CodeBuild computational power would help 🤔 ?

Copilot uses BUILD_GENERAL1_SMALL for compute type, along with the fact it's a linux container, it should map to the "Linux Small" in this doc.

Would you like to change your "compute type" to a larger one, e.g. "Linux Large", and then manually trigger the build to see if it goes through?

To manually trigger a build, you could go to the CodePipeline console, and then hit "Retry" on your build stage.

Edit: this is to help us understand whether the issue can be resolved by increasing CodeBuild's computational resource - a step to diagnose. If this indeeds resolves the issue, then we understand how Copilot could be improved in order to help with use cases like yours.

[Kimblis]

Hey, I tried to increase compute to "Linux Medium" and it was enough. As I see it 2x the price of linux small. I was a bit surprised that linux small was not enough for these few tests. However, then I realised that I also cloning my microservices from github, they also take space, so probably it would make sense that linux small would run out of memory, right? Probably some option in codebuild/pipeline that specifies linux container size would be awesome, right? You could also add to the docs, that if client runs out of memory, he should increase linux container size, and ofc if they could do that without going to aws console, that would also be awesome, right? :D

[jakequalia]

+1 this would be very useful to add :)

[huanjani]

Related: #1516

[ytanay]

We too would find this very useful, currently we have to remember to manually reset the build size in the console after deploying changes to the pipeline - which we invariably notice when builds start timing out after every pipeline change :)

[Lou1415926]

@ytanay The good news is that we are planning to support copilot pipeline override, and then you should be use YAML patch to configure this option ☺️

Related: #4208 #4209 #5034

[huanjani]

Our new pipeline override feature has been released in v1.29.0: https://github.com/aws/copilot-cli/releases/tag/v1.29.0!!

Here is a YAML patch that changes the size to large:
https://gist.github.com/Lou1415926/bd53c0af7299da55f58077736e92729b

[github-actions]

This issue is stale because it has been open 60 days with no response activity. Remove the stale label, add a comment, or this will be closed in 14 days.

[github-actions]

This issue is closed due to inactivity. Feel free to reopen the issue if you have any further questions!