OONI Probe CLI v3.17.0-alpha.1

Overview

This release should be the solid starting point for the 3.17.x train of releases. We will test it next week. If everything is fine, we can then move forward with creating v3.17.0 and with updating all the apps.

What's Changed

✨✨✨ We rewrote the build procedure in Go. We did this for two reasons. The first reason is unit testing. We can now be confident that the build procedure invokes the correct commands. The second reason is that we wanted to compile tor and its dependencies for Android using hardening flags. We implemented these changes in these pull requests:

• feat: simplify writing tests for code using shellx by @bassosimone in #1042
• refactor: factor common code to check whether file exists by @bassosimone in #1045
• feat(shellx): support incremental command construction by @bassosimone in #1046
• feat(runtimex): add the TryN functions by @bassosimone in #1047
• feat(must): convenience package with "must" functions by @bassosimone in #1048
• feat: rewrite darwin build rules in Go by @bassosimone in #1049
• feat: rewrite windows build rules in Go by @bassosimone in #1050
• feat: rewrite generic build rules in Go by @bassosimone in #1051
• feat: rewrite linux build rules in Go by @bassosimone in #1044
• feat: mechanism compile and embed tor by @bassosimone in #1052
• feat: rewrite the android build in Go by @bassosimone in #1053
• feat: convert ios build to use Go by @bassosimone in #1054
• feat: build for Android using Go by @bassosimone in #1055
• cleanup: ghgen should be in internal/cmd by @bassosimone in #1056
• refactor(buildtool): unify C deps with other deps by @bassosimone in #1057
• refactor(buildtool): merge cdeps and android build envs by @bassosimone in #1058
• refactor(buildtool): use naming reminding of bash variables by @bassosimone in #1059
• refactor(buildtool): improve how we set environment variables by @bassosimone in #1060
• feat: cross-compile tor for android using the Go builder by @bassosimone in #1061
• feat(buildtool): add unit tests for android builds by @bassosimone in #1062
• feat: use internal/libtor for android builds by @bassosimone in #1063
• chore: update to ooni/go-libtor v1.1.7 by @bassosimone in #1064
• doc(buildtool): explain ANDROID_NDK_HOME for android gomobile by @bassosimone in #1066

✨✨✨ We continued to improve and modernize the code that communicates with the OONI backend. In particular, in this release, we added new code that allows us to perform A/B testing of the Web Connectivity implementation. This change will allow us to start testing the new Web Connectivity implementation. We will take advantage of this functionality to start implementing and testing throttling measurements inside Web Connectivity, which we will work on soon. In addition, the changes we implemented simplify providing more complex inputs to experiments. (The fact that we can tell the probe which version of Web Connectivity to run is indeed a special case of providing more complex input.) We did this work in:

• refactor: rename check-in nettests results for clarity by @bassosimone in #1036
• refactor(session): observe full check-in response by @bassosimone in #1037
• fix(httpapi): don't return nil, nil with null JSON input by @bassosimone in #1038
• feat: enable A/B testing using check-in API by @bassosimone in #1039

✨✨✨ We added support for exposing build information. This gives us extra confidence about the specific version of the measurement engine being used for collecting a measurement, which is great for data quality assessments. We did this work in:

• feat: expose build information by @bassosimone in #1030

🐛🐛🐛 We fixed a bug where Web Connectivity could be tricked to fetch from localhost:

• fix([email protected]): stop fetching from localhost by @bassosimone in #1027

🚧 🚧 🚧 We completed a long standing refactoring by moving internal/engine packages to internal. It's good to do this right before making a new release train (3.17.x) because it simplifies backporting patches if needed:

• refactor: prepare for renaming all experiments by @bassosimone in #1032
• refactor: continue moving packages outside of internal/engine by @bassosimone in #1031
• refactor: move all experiments into internal/experiment by @bassosimone in #1033
• cleanup: finish moving packages away from engine by @bassosimone in #1034
• chore: update readme files and perform minor cleanups by @bassosimone in #1035

🚧 🚧 🚧 We did the usual grunt work associated with preparing a release: updating dependencies and cleaning up stuff. We specifically did this work in the following pull requests:

• cleanup: use sync/atomic instead of atomicx by @bassosimone in #1028
• chore: use go1.19.5 by @bassosimone in #1029
• feat(engine): make netxlite.utlsConn public by @kelmenhorst in #1018
• chore: update dependencies by @bassosimone in #1041
• fix: attempt to make all workflows green by @bassosimone in #1065

Full Changelog: v3.17.0-alpha...v3.17.0-alpha.1