Releases · ooni/probe-cli

01 Dec 11:41

v3.19.2

85c6d1b

OONI Probe CLI v3.19.2

This is a stable release. On a high-level, this is what changed in 3.19:

✨ Better Support for Measuring Throttling
✨ Introducing OONI Probe Bridges
✨ Improving Measurements Scrubbing
✨ Netemx: A Better Unit/Integration Testing Framework
✨ OONI Run v2 Support
✨ Unit Testing Improvements (go test -short ./... does not use the host network anymore)
🐛 OONI Probe Bootstrap fixes
🐛 Web Connectivity v0.4 Fixes
🐛 Snowflake Fixes

We recommend reading v3.19.0-alpha's changelog because it contains an exhaustive description of the most significant changes in 3.19 compared to 3.18. The following is the complete list of all the releases that led to this release:

On top of the v3.19.1 release, this follow-up release adds the following changes backported from the master branch:

🐛 fix(signal): trim down tested endpoints 85c6d1b

Assets 45

21 Nov 14:41

github-actions

v3.19.1

96e96be

OONI Probe CLI v3.19.1

This is a stable release. On a high-level, this is what changed in 3.19:

✨ Better Support for Measuring Throttling
✨ Introducing OONI Probe Bridges
✨ Improving Measurements Scrubbing
✨ Netemx: A Better Unit/Integration Testing Framework
✨ OONI Run v2 Support
✨ Unit Testing Improvements (go test -short ./... does not use the host network anymore)
🐛 OONI Probe Bootstrap fixes
🐛 Web Connectivity v0.4 Fixes
🐛 Snowflake Fixes

On top of the v3.19.0 release, this follow-up release adds the following changes backported from the master branch:

🐛 Update signal test endpoints 6b32fbf

Assets 45

20 Oct 09:11

github-actions

v3.19.0

3cce54b

OONI Probe CLI v3.19.0

This is a stable release. On a high-level, this is what changed in 3.19:

✨ Better Support for Measuring Throttling
✨ Introducing OONI Probe Bridges
✨ Improving Measurements Scrubbing
✨ Netemx: A Better Unit/Integration Testing Framework
✨ OONI Run v2 Support
✨ Unit Testing Improvements (go test -short ./... does not use the host network anymore)
🐛 OONI Probe Bootstrap fixes
🐛 Web Connectivity v0.4 Fixes
🐛 Snowflake Fixes

On top of the v3.19.0-alpha.5 release, this follow-up release adds the following changes backported from the master branch:

🐛 fix(registry): disable vanilla_tor by default 6a40751
🚧 chore: upgrade NDKVERSION to latest r26 09d3102
🚧 chore: set version to 3.19.0 3cce54b

In short, we also marked vanilla_tor as disabled by default, because it has issues on Android. We'll enable it again using the check-in API, but we would like to take advantage of the possibility of deciding to disable it in the future. We also need to upgrade the NDK since there has been a new release of the r26 LTS.

Assets 45

18 Oct 16:40

github-actions

v3.19.0-alpha.5

7c4e537

OONI Probe CLI v3.19.0-alpha.5 Pre-release

Pre-release

This release is an alpha release. It is not ready for general availability and should only be used for QA.

A stable release will follow suit hopefully in a few days from today.

This release is based on v3.19.0-alpha.4. Make sure you also read the changelog of:

On top of the v3.19.0-alpha.4 release, this follow-up release adds the following changes backported from the master branch:

🐛 fix(nettests): failing to start a nettest is a warning 7c4e537

In short, without this diff, running experimental with default-disabled experiments such as riseupvpn when they are check-in-disabled shows a visual error that modifies the view and which is totally unnecessary to show.

Assets 45

16 Oct 22:40

github-actions

v3.19.0-alpha.4

a2b71cb

OONI Probe CLI v3.19.0-alpha.4 Pre-release

Pre-release

This release is an alpha release. It is not ready for general availability and should only be used for QA.

A stable release will follow suit hopefully in a few days from today.

This release is based on v3.19.0-alpha.3. Make sure you also read the changelog of:

On top of the v3.19.0-alpha.3 release, this follow-up release adds the following changes backported from the master branch:

🐛 fix(ios): publish libcrypto, libssl, libevent, libtor, libz frameworks a2b71cb

In short, this release fixes the iOS build by publishing all the required dependencies frameworks.

Assets 45

12 Oct 18:14

github-actions

v3.19.0-alpha.3

4513a12

OONI Probe CLI v3.19.0-alpha.3 Pre-release

Pre-release

This release is an alpha release. It is not ready for general availability and should only be used for QA.

A stable release will follow suit hopefully in a few days from today.

This release is based on v3.19.0-alpha.2. Make sure you also read the changelog of:

On top of the v3.19.0-alpha.2 release, this follow-up release adds the following changes backported from the master branch:

🚧 refactor(buildtool): use list for OpenSSL ./Configure flags 06cda7b
🚧 cleanup(buildtool): disable unit tests when building tor cfb3c85
🐛 fix(buildtool): let libevent use OpenSSL's pkgconfig 79740bc
✨ feat(buildtool): build zlib, openssl, libevent, and tor for iOS ed915b5
🚧 chore(buildtool): add ios-build unit tests d71e416
✨ feat(ios): replace go-libtor with ./internal/libtor 4513a12

In short, this release includes work to build tor for iOS using the same build tool used for Android.

Assets 35

11 Oct 20:58

github-actions

v3.19.0-alpha.2

0eabe59

OONI Probe CLI v3.19.0-alpha.2 Pre-release

Pre-release

This release is an alpha release. It is not ready for general availability and should only be used for QA.

A stable release will follow suit hopefully in a few days from today.

This release is based on v3.19.0-alpha.1. Make sure you read the changelog of v3.19.0-alpha.1 and v3.19.0-alpha.

On top of the v3.19.0-alpha.1 release, this follow-up release adds the following changes backported from the master branch:

🚧 cleanup(riseupvpn): remove summary keys 7c45887
🚧 refactor(riseupvpn): rename structs, change progress, bump version cafc118
🚧 refactor(riseupvpn): handle failing API and simplify test 3989fd4
🚧 feat(riseupvpn): include bootstrap in progress a1b7f60
🚧 chore(ooniprobe): add riseupvpn to experimental test suite 0eabe59

In short, this release includes work to re-enable the riseupvpn experiment as part of ooniprobe run experimental. We are adding back this experiment as a data collection experimental experiment. We are not performing local classification of the results in the probe anymore. Also, this experiment is disabled-by-default, which means we can choose when to run it using the check-in API. We will slowly reintroduce this experiment to all users and monitor the results it is collecting as part of the 3.19 cycle. Further improvements to this experiment will be implemented in subsequent ooniprobe releases.

It's important to note that riseupvpn data format has changed, and the experiment minor version number has changed.

We now have just two top-level test keys:

api_failures (a []string containing all API calls failures), and
ca_cert_status (a bool which was also present in the previous version of riseupvpn and has not changed its semantics).

If you were processing riseupvpn measurements, you should update your code to use the new keys described above.

Assets 35

11 Oct 16:19

github-actions

v3.19.0-alpha.1

a4037a2

OONI Probe CLI v3.19.0-alpha.1 Pre-release

Pre-release

This release is an alpha release. It is not ready for general availability and should only be used for QA.

A stable release will follow suit hopefully in a few days from today.

This release is based on v3.19.0-alpha, whose changelog you are encouraged to read.

On top of the v3.19.0-alpha release, this follow-up release adds the following changes backported from the master branch:

✨ feat: mvp of conditionally enabling experiments 85b63eb
🐛 fix(torsf): bump the version number e5f950b
✨ feat: add echcheck to the experimental suite a8f4fc2
🐛 fix(ooniprobe): make sure we actually run echcheck d13a7ca
🐛 fix(registry): mark torsf as disabled by default 7bcf415
🚧 chore: use go1.20.10 a4037a2

In short, we're now based on go1.20.10 (released yesterday). Also, the echcheck experiment now runs as part of ooniprobe run experimental. However, and notably, both echcheck and torsf are now disabled-by-default experiments. This means that they will only run if in the last 24h the backend has told us it was safe to do so. (You can always bypass this restriction by setting the OONI_FORCE_ENABLE_EXPERIMENT environment variable to 1.)

We're setting echcheck as disabled by default because it is a new experiment and we want to use the backend API to incrementally and gradually expose this experiment to all users. We're setting torsf as disabled by default because Snowflake made changes during the ooniprobe 3.18 cycle that broke the torsf experiment, and we want to be robust to potential future similar changes. Hence, while as part of this release we will configure the OONI backend API to allow all users to run torsf, we reserve the possibility of disabling torsf, should it be needed, anytime in the future.

Assets 35

10 Oct 16:36

github-actions

v3.19.0-alpha

29e8c88

OONI Probe CLI v3.19.0-alpha Pre-release

Pre-release

Summary

This release is an alpha release. It is not ready for general availability and should only be used for QA.

A stable release will follow suit hopefully in a few days from today.

On a high-level, these are the most important changes:

✨ Better Support for Measuring Throttling
✨ Introducing OONI Probe Bridges
✨ Improving Measurements Scrubbing
✨ Netemx: A Better Unit/Integration Testing Framework
✨ OONI Run v2 Support
✨ Unit Testing Improvements (go test -short ./... does not use the host network anymore)
🐛 OONI Probe Bootstrap fixes
🐛 Web Connectivity v0.4 Fixes
🐛 Snowflake Fixes

Read on for more specific details!

✨ New Features

Better Support for Measuring Throttling

These changes implement a better support for measuring throttling where we periodically collect download speed samples during the lifetime of downloads performed using Web Connectivity (or any other network experiment using measurexlite):

✨ chore: import memoryless from m-lab/go by @bassosimone in #1163
✨ feat: lightweight throttling measurements by @bassosimone in #1166
✨ feat(dslx): collect speed samples by @bassosimone in #1167

Note that, because these changes are measurexlite specific, and because only Web Connectivity LTE uses measurexlite, we would only get the benefit of better throttling measurements once we're fine switching all users to use Web Connectivity LTE. (Currently, 99% of the users are still using Web Connectivity v0.4, since we still have to finish doing A/B testing to make sure that the new version of Web Connectivity is not introducing any regressions compared to version v0.4.)

Introducing OONI Probe Bridges

We added support for OONI Probe Bridges. (We initially named this feature "beacons", which is why many pull requests below mention "beacons".) A OONI Probe Bridge is a host that allows us to connect to the OONI backend or test helpers. The current implementation knows about a single bridge, but it is possible to add more bridges (and we will add more bridges). The important concepts are that the probe knows the bridge IP address in advance and that there's the guarantee we can use pretty much any SNI with the bridge itself, even though, obviously, we use the api.ooni.io verify hostname when verifying the certificate. By default OONI Probe tries using a bunch of predefined SNIs and remembers what works to use it consistently. This change currently only modifies how we create TLS connections with the OONI backend, but we plan on extending the scope of these changes further in subsequent releases.

There is also a mechanism to bypass the probe behavior and force it to use a specific bridge. We're documenting this mechanism here with the understanding that this functionality should be considered as a technical preview and may change in future releases without any need on our end to change the major version number. With that caveat out of the way, with this release of OONI Probe, if you create $HOME/.ooniprobe/engine/bridges.conf, it will be used to override how we communicate with the backend. The file format is JSON with comments, and here is an example:

{
  // The DomainEndpoints field maps a domain's endpoint to a list of bridges
  "DomainEndpoints": {
    "api.ooni.io:443": [

      // A bridge is characterized by an IP address, a Port, a SNI to send on the wire, the
      // hostname to use for TLS certificate verification and a delay.
      //
      // The delay can be useful to stagger dial attempts such that you do not try all
      // the bridges at the same time. The unit of InitialDelay is nanoseconds.
      {
        "Address": "130.192.91.211",
        "InitialDelay": 0,
        "Port": "443",
        "SNI": "www.example.com",
        "VerifyHostname": "api.ooni.io"
      }

    ]
  }
}

There is also a new file $HOME/.ooniprobe/engine/httpsdialer.state that keeps the on-disk state used by OONI Probe to remember which bridges worked. This file also includes information about some bridges that did not work, however we periodically prune this cache to avoid it growing too large in case there are many failures. (We're more interested on what works anyway.)

✨ feat(enginenetx): add configurable HTTPS dialer by @bassosimone in #1283
✨ feat(testingx): add code to ensure we close all conns by @bassosimone in #1284
✨ feat(enginenetx): make sure HTTPSDialer closes all connections by @bassosimone in #1285
✨ fix(enginenetx): pass context to tactics callbacks by @bassosimone in #1286
✨ refactor: adapt to netem pinning certificates to hosts by @bassosimone in #1287
✨ refactor(enginenetx): split https dialer implementation by @bassosimone in #1289
✨ feat(enginenetx): introduce loadable TLSDialer policy by @bassosimone in #1290
✨ refactor(enginenetx): introduce stats and make tactic a struct by @bassosimone in #1291
✨ fix(enginenetx): store endpoint into the tactic by @bassosimone in #1292
✨ refactor(enginenetx): rename HTTPTransport to Network by @bassosimone in #1293
✨ feat: add github.com/tailscale/hujson extensions by @bassosimone in #1294
✨ feat(enginenetx): use the new HTTPSDialer by @bassosimone in #1295
✨ fix(enginenetx): refine the happy-eyeballs algorithm by @bassosimone in #1296
✨ refactor(enginenetx): make static/loadable policy easier to use by @bassosimone in #1297
✨ feat(enginenetx): honor user-provided policy by @bassosimone in #1298
✨ feat(enginenetx): track operations and collect stats by @bassosimone in #1299
✨ refactor(enginenetx): make LookupTactics async by @bassosimone in #1300
✨ fix(enginenetx): stabilize happy eyeballs algorithm by @bassosimone in #1301
✨ feat(enginenetx): introduce beacons policy by @bassosimone in #1302
✨ refactor(enginenetx): make beacons API private by @bassosimone in #1303
✨ refactor(enginenetx): make stats API private by @bassosimone in #1304
✨ feat(enginenetx): prune old entries from stats by @bassosimone in #1305
✨ refactor(enginenetx): store address and port separately by @bassosimone in #1306
✨ refactor(enginenetx): group by domain's endpoint by @bassosimone in #1307
✨ refactor(enginenetx): make static-policy API private by @bassosimone in #1308
✨ refactor(enginenetx): make dns-policy API private by @bassosimone in #1309
✨ refactor(enginenetx): make https-dialer API private by @bassosimone in #1310
✨ feat(enginenetx): support getting stats on a domain endpoint by @bassosimone in #1311
✨ feat(enginenetx): add policy based on stats by @bassosimone in #1312
✨ feat(enginenetx): enable the stats-based policy by @bassosimone in #1313
✨ chore(enginenetx): more tests and robustness checks by @bassosimone in #1314
✨ fix(enginenetx): use dns policy with proxy (+renames) by @bassosimone in #1315
✨ fix(enginenetx): gracefully handle more nil cases by @bassosimone in #1316
✨ fix(enginenetx): periodically trim statistics by @bassosimone in #1317
✨ feat(enginenetx): extend beacons policy for THs by @bassosimone in #1318
✨ fix: rename beacons to bridges by @bassosimone in #1349

Support for HTTP/HTTPS proxies

As part of work to introduce OONI Probe bridges, we also removed limitations in the codebase that prevented to use HTTP or HTTPS proxy URLs with ooniprobe --proxy URL and miniooni --proxy URL. We will update mobile apps to allow using this kind of proxies as well. Before these changes, we only supported (a) SOCKS5 proxies and (b) creating proxies using Psiphon, vanilla tor, or tor along with snowflake (respectively, --proxy socks5://<addr>:<port>, --proxy psiphon:///, --proxy tor:///, --proxy torsf:///). Assuming you have, e.g., tor exposing a proxy on port 9080, now you will be able to use ooniprobe --proxy http://127.0.0.1:9080/.

✨ chore: run go fmt ./... by @bassosimone in #1272
✨ feat(testingx): introduce more comprehensive HTTP(S) proxy by @bassosimone in #1274
✨ feat(testingproxy): test HTTP(S) proxies using netem by @bassosimone in #1275
✨ feat(testingx): more tests for NewHTTPProxyHandler by @bassosimone in #1276
✨ cleanup: use testingx.NewHTTPProxyHandler as proxy by @bassosimone in #1277
✨ feat(UnderlyingNetwork): add support for ListenTC...

Contributors

bassosimone, imw, and 3 other contributors

Assets 35

18 Jul 16:07

github-actions

v3.18.1

4eff308

OONI Probe CLI v3.18.1

This patch release includes the following changes since v3.18.0:

✨ webconnectivity LTE: better throttling support e1a63d1 5c1b7d2
🚧 oonimkall: experimental OONI Run v2 API 4eff308
🚧 measurexlite: do not depend on tracex 527d115
🚧 update to go1.20.6 394a273
🚧 update to 2023-07 geoip databases ae93c56
🚧 update all the other deps we can easily update 2d91295

Full Changelog: v3.18.0...v3.18.1

Assets 35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Summary