Finalize egress work

[cantsin]

As part of Tock's compliance process, the capability for egress filtering is set up for cloud.gov deployments of Tock. We have set up egress spaces for both staging (which should always be on) and production (not on, but available), but more work needs to be done before we finalize egress for the entire stack:

• Egress functionality does not seem to be stable. Production/staging sometimes breaks (for the former, maybe because of recycle-prod) and we'd like to do more testing as to why
• Someone should run through the documentation and make sure it looks sane (especially the cloud.gov documentation), including re-creating egress from scratch in the space and adding more documentation around debugging (e.g., making sure caddy works locally)
• We need more documentation on how to update and check the status of egress filtering
• Because of the nature of egress, which involves sensitive variables, we don't have a good way to actually document our egress setup other than via documentation -- we could look at other projects and see how they address this

(added by @jduss4 )

• Investigate removing the public-egress policy from Tock staging. Currently the app fails to restage when this policy is removed due to connection errors related to New Relic (possibly related to Enable New Relic connections GSA-TTS/cg-egress-proxy#15) (a possible complication: staging: fix new relic ssl cert errors #1661)

[juliaklindpaintner]

Consider turning this into an epic for TLC crew purposes — will follow up!

[alexbielen]

@cantsin and @edwintorres :

Are you still looking for TLC Crew to help on this next increment (Dec 11 - 25)?

[edwintorres]

@alexbielen yes that's correct

[jduss4]

I put in a little work running through the steps in the existing documentation and adding more to it here: #1706

[jduss4]

Unfortunately I was not able to complete this card before the break. However, during this time we the following:

1. Built out the documentation
2. Redeployed the staging versions of egress + tock and confirmed the behavior works for restricting egress
3. Experimented (unsuccessfully) with removing the public-egress application security group from tock staging and identified a new area for investigation in the setup for New Relic

[cantsin]

I think we can finally close this issue :) If anything comes up with egress in the future, I'll make a separate ticket. Many thanks to everyone involved!!