Make Elastic Serverless Forwarder available in AWS GovCloud regions

[ravikesarwani]

Currently, Elastic Serverless Forwarder(ESF) is not available in AWS GovCloud regions.

From the AWS docs "Applications that are publicly shared in other AWS Regions are not automatically available in AWS GovCloud (US) Regions. To make applications available in AWS GovCloud (US) Regions, you must publish and share them independently of other AWS Regions".

This issue is to publish ESF in AWS SAR to be made available on the AWS GovCloud(US) Regions.

There are certain requirements around who can have access to AWS GovCloud(US) Regions. For example:
"AWS GovCloud (US-East) and (US-West) Regions are operated by employees who are U.S. citizens on U.S. soil. AWS GovCloud (US) is only accessible to U.S. entities and root account holders who pass a screening process. Customers must confirm that they will only use a U.S. person (green card holder or citizen as defined by the U.S. Department of State) to manage and access root account keys to these regions."

• Figure out the logistics of AWS account that can be used for this
• Figure out the logistics of who can get access and get access to those folks
• Figure out the process of publishing ESF from that account to AWS GovCloud (US-East) and (US-West) Regions
• Define the process of how we will maintain this for each updated version over time

Related

Preview Give feedback

1. https://github.com/elastic/enhancements/issues/17413
2. https://github.com/elastic/enhancements/issues/18231

[michaelmagyar]

Adding some thoughts as Functionbeat is deprecated with support ending in less than a year, and federal clients and their service providers will need extensive time to plan, implement, re-document, and possibly get re-audited on the change (i.e., they may have to add SQS, Secrets Manager, etc. to their packages if they weren't using them before).

Elastic currently has a GovCloud account cluster that supports the FedRAMP offering. Because Elastic is already approved for GovCloud, the team that handles that account cluster should be able to either add ESF publishing directly to the existing GovCloud account(s) or create a new GovCloud account specifically for ESF publishing. The issue is compliance, and that needs to be thought of first in this case.

It may be possible to immediately add ESF to the existing GovCloud accounts. However, there might be some compliance issues with immediately adding additional functionality to that environment because that might change the ATO package and require a significant change request/re-audit.

If that is going to take some time (it could easily be 6-12 months), then an alternative is to:

• Now: Have a separate GovCloud account provisioned just for publishing ESF to the GovCloud SAR
• Later: Add ESF to the FedRAMP compliance package and migrate it to the existing federal AWS GovCloud accounts
• Even Later: Deprecate the initial GovCloud SAR registry publication and recommend customers switch to the new ARN

If that flow ends up happening and ESF is published to GovCloud outside of the existing federal offering, some entities still may not be able to leverage it for compliance reasons, but they would at least have the ability to make the risk-based decision and/or test that it works with time to implement it properly.

I think the steps are likely:

• Coordinate with the Elastic federal team to determine when ESF will become part of the federal offering (does it require a re-audit or can it be added immediately)
• If not immediately, consider requesting another GovCloud account that will remain outside of the FedRAMP package even if just for temporary ESF publishing to fix any compatibility issues with GovCloud
• Set up the existing pipeline to additionally publish to the chosen GovCloud account
• Tweak the ESF code that is not compatible with GovCloud (e.g., ARNs need to be "arn:${AWS::Partition}:" instead of "arn:aws:")

I hope this happens quickly and that it can be immediately added to the existing FedRAMP accounts/package, but I am not optimistic.

[aspacca]

@michaelmagyar I'd need to pair with someone with access to the GovCloud account cluster with the proper permission to publish an app on SAR.

They'll have to publish the forwarder and maintain it on the GovCloud account. From the technical requirements there should not be much to do, but I cannot do anything on my side without pairing with them and transfer them the knowledge they require.

The same if another GovCloud account is requested.

Let's just arrange the proper point of contact and I don't see, from the technical point of view, any great blockers on this.

[michaelmagyar]

I see that this issue has not been resolved for almost a year. What is the current plan to support serverless log forwarding in GovCloud? Go Lambda support is being discontinued at the end of this year, so functionbeats will no longer be viable, even as a deprecated option, without a lot of changes.

Is the plan really to force GovCloud clients to add VMs just to collect logs? What about clients that don't currently use/authorize VMs in their environment?

[bturquet]

Hey @michaelmagyar, we still have the plan to support serverless log forwarding in GovCloud. We are working with Legal Team to have the Sponsor approval to make it happen.

More info here: https://github.com/elastic/infosec/issues/14266

[michaelmagyar]

Hello Elastic. Can you please provide an update to this? I see that PR #510 was merged. However, although there is a published ESF package on AWS SAR, the publisher is not verified and the helper/nested applications are missing.

What is the current status/timeline?

We were told that our current spend of 5 figures a month is not large enough to have access to a technical account manager except for renewals, so this appears to be the only route for us to get updates on this outside of support.

Note: functionbeat is no longer deployable given that AWS has disabled the Go runtime for new functions, so continuing to use that would require rebuilding the package using AL2 runtime and adding Go on top.

[bturquet]

Hi @michaelmagyar , we are still waiting for internal permissions to be granted for us, to deploy last ESF version in GovCloud SAR. The ETA is in 2 weeks.

• https://github.com/elastic/enhancements/issues/17413
• https://github.com/elastic/enhancements/issues/18231

[michaelmagyar]

Hello @bturquet , are there any updates on the ETA for ESF in GovCloud?

[aspacca]

hi @michaelmagyar , we are still blocked on having proper access to GovCloud in order to make ESF available there.
We'll send an update next week

[kaiyan-sheng]

@michaelmagyar I was able to get the proper permission this week and I changed helper applications to public. So you should be able to see them now.

However, although there is a published ESF package on AWS SAR, the publisher is not verified and the helper/nested applications are missing.

For the publisher not being verified, we are still working with the AWS side to get that solved.

We have not published a new version of ESF yet due to some other issues. Will update this ticket once we are able to publish a new version.

[kaiyan-sheng]

Last time I had proper permission to access govcloud, I was able to change ESF helper applications to public. But since then, we had multiple releases so we should push the latest release to govcloud and also might be a good time to start looking into how we can automate this process.

[zmoog]

Last time I had proper permission to access govcloud, I was able to change ESF helper applications to public. But since then, we had multiple releases so we should push the latest release to govcloud and also might be a good time to start looking into how we can automate this process.

Great! We can set some time aside to document the process. I'll send you an invite.