Moving from requirements.txt to poetry

[endorama]

Describe the enhancement: Change dependency management to a solution that simplifies and enhance dependency management.

Describe a specific use case for the enhancement or feature: First let me say that I appreciate the current solution, it works and solves the basic need of the project.

I think we can improve on top of it to help streamline the development, in both onboarding and maintenance.

These are the drawbacks I see in the current implementation:

• we need multiple requirements file based on the environment, this complicates the setup and maintenance; this would also prevent confusion like in Fix CI test errors #203 (comment)
• there is no Python version requirement
• pip freeze > requirements.txt makes more complex to update dependencies (and requires manual work to split dependencies in separate files)
• there is no dependency integrity check (like package shasum checks)

Poetry would allow us to improve on the current situation on these 4 fronts.

@aspacca @girodav what do you think?

/cc @elastic/obs-cloud-monitoring

[aspacca]

@endorama I'm good on this, we just need to be sure that we export to requirements.txt with poetry for sam build(https://github.com/elastic/elastic-serverless-forwarder/blob/main/.internal/aws/scripts/dist.sh#L78)

similar to what's done here: https://gist.github.com/jakebrinkmann/ccd3537a61894fb676b10fa50fc2f9a3#file-readme-md