Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EBSI digital identity DDoS hardening using IP reputation #7481

Open
synctext opened this issue Jun 12, 2023 · 46 comments
Open

EBSI digital identity DDoS hardening using IP reputation #7481

synctext opened this issue Jun 12, 2023 · 46 comments
Assignees

Comments

@synctext
Copy link
Member

synctext commented Jun 12, 2023

Survey Q1 + Starting thesis 10 jan {job next to thesis}

Also interested in the problem of online trust. General intro and overview by Bruce {rockstar of security research} Ten Risks of PKI: What You’re not Being Told about Public Key Infrastructure. Technical discussion on HackerNews on passport failures and public key directories.

{duplicate from # 7423}

Topic Title of paper and URL
strong identity A Truly Self-Sovereign Identity System
strong identity TrustVault: A privacy-first data wallet for the European Blockchain Services Infrastructure
strong identity Zero-Trust Architecture for Legal Entities
strong identity Distributed Attestation Revocation in Self-Sovereign Identity

More on passport-level digital identity. Warning given 😸 "this is complex stuff". identity is the foundation of trust in the old analog world. Please make a reading list about SSI, EBSI, EBP and eIDAS2

please note the survey @ Tribler lab methodology: https://github.com/Tribler/tribler/wiki/MasterThesis#10-ects-literature-survey-at-tribler-lab (e.g. try out all known open source wallets for suvey and grade them with stong/weak analysis in a table with screenshots { https://walt.id/ebsi ; https://github.com/walt-id })

@AdiDumi
Copy link

AdiDumi commented Jun 12, 2023

why has public key infrastructure failed for 35 years

@synctext
Copy link
Member Author

synctext commented Aug 28, 2023

Ask Google Scholar about PKI failure. First result:
https://www.nics.uma.es/pub/papers/JavierLopez2005.pdf
With the upcoming European passport this given a nice modern twist to 35(?) years of failure: Failure of Public Key Infrastructure for 35 years: lessons for the European passport-grade digital identity (EBSI) {putting 2 storylines into 1 title}. Shorter attempt: 35 years of failure: European Digital Identity lessons

see 2016 ideas on Self-sovereign identity (SSI). TUDelft Master student full-time on EU digital ID from 2021. Discussed the Literature Survey wiki documentation

{brainstorm in bit harsh term} A fraud-resilient authentication method is notoriously hard to realise. Academic thinkers have offered little help, academic literature extensively documents numerous ideas and design sketches. Public key cryptography was invented in the year: ... In the 35 years since this invention we have failed to utilise this invention. The European Digital Identity project (EBSI) is only the latest in a long line of failed attempts. For instance, "Overview of the German identity card project and lessons learned (2020 update)" and "The giant is lagging behind - How the German electronic ID fails to reap its potential.
This survey lists the numerous projects to provide digital identity and also highlights the decades scientific research. To date, no solution exists for the 400 million citizens within the EU. Finally, we list the vital lessons for the upcoming project to provide an EU-wide passport-grade digital identity. Contrary to numerous past projects, a refreshing amount of transparency is provided. For instance, see the detailed public EBSI node operator operational handbook.

Bit of general info news article "Digital Identity: Where We Began, Where We Are And Where We Are Going"

{early brainstorm for master thesis} The above builds expertise on wide scope of upcoming EU digital ID. Security will we essential, but EBSI server is based upon IBM Hyperledger technology. This is expected to only serve a good purpose during development. For full-scale production usage as the underpinning of the entire EU digital economy Hyperledger server probably needs replacing. Ideal outcome would be re-using the infrastructure at Delft to develop a 12-line Kotlin script to bring down such a EBSI Hyperledger server. See 2021 Delft master student who made the first open source Android EBSI v2 communications lib. Good news, save this project with application-level firewall rules????

@AdiDumi
Copy link

AdiDumi commented Aug 28, 2023

On the risk of misbehaving RPKI authorities

REGULATION (EU) on eid
'Building trust in the online environment is key to economic and social development. Lack of trust, in particular because of a perceived lack of legal certainty, makes consumers, businesses and public authorities hesitate to carry out transactions electronically and to adopt new services.'

@AdiDumi
Copy link

AdiDumi commented Sep 20, 2023

Survey Template

@synctext
Copy link
Member Author

synctext commented Oct 11, 2023

@synctext
Copy link
Member Author

synctext commented Nov 1, 2023

@AdiDumi
Copy link

AdiDumi commented Nov 22, 2023

Almost final draft
Missing abstract, conclusion and some table information
Literature_Survey_IN4306.pdf

@synctext
Copy link
Member Author

synctext commented Nov 22, 2023

image

  • J.H. Ellis, Communications - Electronics Security Group, Government Communications Headquarters, Research Report No. 3006, The Possibility of Secure Non-Secret Digital Encryption, January 1970. Secret.. the possibility of non-secret crypto is from Jan 1970, 53 years ago

  • Just stated that it might be possible, no algorithm for one-way function, no implementation. That was RSA.

  • Missing milestone: Trust-on-first-use (TOFU, 2008) Perspectives: Improving SSH-style host authentication with multi-path probing. low-cost and simple key management model

  • small errors: "W. Diffie; M. Hellman" wrote the [27] citation. That's just key agreement, not the invention or realisation of public key crypto.

  • typos: \event{1996}{SDSI intorduction}

  • Dan Boneh, FEBRUARY 1999: twenty years of attacks on the rsa cryptosystem Essence: securely implementing RSA is a nontrivial task. We conclude: deploying public key cryptography such as RSA is also a non-trivial task!

  • "III. A HISTORY OF ALTERNATIVES", more like "evolution of PKI"?

  • "TABLE I: Overview of the ranking countries eID." the core taxonomy table! Facinating stuff, please at least double the number of countries+failures

  • EU on ID. "Every time an App or website asks us to create a new digital identity or to easily log on via a big platform, we have no idea what happens to our data in reality. That is why the Commission will propose a secure European e-identity. One that we trust and that any citizen can use anywhere in Europe to do anything from paying your taxes to renting a bicycle. A technology where we can control ourselves what data is used and how." Ursula von der Leyen, President of the European Commission, in her State of the Union address, 16 September 2020

  • {early brainstorm for master thesis - PART II} EBSI wallet, wallet-to-wallet communication (known IPv4), overlay (key lookup),data sharing (EBSI-to-EBSI), server hardening!
    (repeating)The above builds expertise on wide scope of upcoming EU digital ID. Security will we essential, but EBSI server is based upon IBM Hyperledger technology. This is expected to only serve a good purpose during development. For full-scale production usage as the underpinning of the entire EU digital economy Hyperledger server probably needs replacing. Ideal outcome would be re-using the infrastructure at Delft to develop a 12-line Kotlin script to bring down such a EBSI Hyperledger server. See 2021 Delft master student who made the first open source Digital Basic Identity - EU EBSI gateway and "regie op gegevens" #6023 (comment). Good news, save this project with application-level firewall rules????

    • time for coding again!
    • Data-vault code + wallet v2 interfacing with EBSI.
    • big milestone: talk to live EBSI server
    • Check out the code: https://ec.europa.eu/digital-building-blocks/code/projects/EBSI

@AdiDumi
Copy link

AdiDumi commented Nov 22, 2023

@AdiDumi
Copy link

AdiDumi commented Dec 12, 2023

Finished final version of the Survey.
Worked on the abstract, conclusions, history of PKI evolution, added countries to the survey and completed the taxonomy table. Rephrasing some paragraphs and added more literature about PKI failures and EU eIDs.
Literature_Survey_IN4306_final.pdf

@synctext
Copy link
Member Author

synctext commented Dec 13, 2023

  • Solid work 🥇
  • Not explicit what PKI offers
  • Difficult to read academic writing style with section opening sentences covering 6 lines sometimes: The core element of the Public Key Infrastructures, key exchange with the RSA cryptosystem, has been the subject of different attacks from its introduction and securely implementing RSA is a nontrivial task, concluding that deploying public key cryptography such as RSA is also a non-trivial task
  • Would love to hear even a small success story, I get depressed from these 53 years of failings; However, only a few have succeeded, and there are several reasons behind the failure of PKIs, which can be categorized into technical, economical, legal, and social factors [37].. See wise people of Reddit: is Keybase dead?. That was the best startup we had in PKI space.
  • cultural reason is missing, digital authentication infrastructure has overlap with the "identity wars". Storage of government attributes like gender 👯‍♂️ with cryptographic authentication are notorious, see the "gender X" UK petition for border-crossing identity documents.
  • Leading company today seems: https://idk.digidentity.eu/Overview/
  • Would be great work for an arXiv upload !!!
  • First steps of master thesis: study code our our existing EBSI wallet, study the server code.

@AdiDumi
Copy link

AdiDumi commented Jan 10, 2024

The survey is uploaded to arXiv with the lastest version.
Literature_Survey_IN4306 (2).pdf

@synctext
Copy link
Member Author

synctext commented Jan 10, 2024

Completed! Today the master thesis work starts 🎉

@AdiDumi
Copy link

AdiDumi commented Jan 24, 2024

Studied the code for existing EBSI wallet in the SuperApp
Studied the server code and also on the gitlab with the APIs and wallets available now
and wallets

@synctext
Copy link
Member Author

synctext commented Jan 24, 2024

Concrete tasks to further explore your thesis focus.

  • Try to find an EBSI (pilot) server to talk to .
    • try for at least days to find one, backup plan: study, compile the EBSI data vault superapp
    • Kotlin tooling
  • goal: build command-line tooling for performance analysis
  • outcome: this server crashes with a workload of 3k `did_write` requests per second. Stable with 2.5k tps.
    • measure response time, jitter
    • bandwidth usage
  • passkeys, FIDO2, webAuth are heavily pushed by Big Tech.
    • only low-level TOFU for email and e-commerce
    • no bank KYC, passport-grade legal signature, or air-plane boarding
    • However, this is usable and will be HUGE. Deployed to all Apple fans
    • authenticator apps with: time-based one-time password (TOTP; specified in RFC 6238) and HMAC-based one-time password (HOTP; specified in RFC 4226)
    • EBSI will almost be forced to interact with this or be compatible somewhat ??? (or given the EU temperature: "EU passkeys Regulation Act", forcing them to change)
  • mental note: this master student working on PKI, athentication and digital identity does not know backup codes
  • Next sprint: 1-page "Problem Description" as MARE.tudelft.nl first stage evaluation doc + also first sample master thesis chapter.
  • Either Rowdy or Egbert can help with EBSI daily details.

@synctext
Copy link
Member Author

@synctext synctext changed the title msc placeholder: solid coding specialist msc placeholder: PKI, EBSI, and SSI Feb 7, 2024
@synctext
Copy link
Member Author

synctext commented Feb 7, 2024

Next week we should get an EBSI server image to install, so there is movement...

@AdiDumi
Copy link

AdiDumi commented Feb 19, 2024

Update sprint:

  • key takes from Qualified Ledgers: Bridging the Gap between Blockchain Technology and Legal Compliance:
    - eIDAS 2.0 shaping the regulatory framework for digital identity and trust services within the European Union with an accent on Decentralized digital identities and ecosystems and Self Sovereign identities
    - qualified electronic ledgers aligns with the principles of eIDAS 2.0
    - (EBSI) and its transition into the European Digital Infrastructure Consortium (EDIC) for decentralized infrastructure for public services across Europe
    - EBSI could evolve into a qualified electronic ledger, enhancing security and reliability for various trust services and with DLT intersect with the requirements of eIDAS 2.0, such as digital credentials, cross-border cooperation, and regulatory compliance.
    - German government's decision not to support DLT for their national EUDI wallet because provable security standards, addressed through certification requirements maybe?

  • Challenges in Stopping Application Layer DDoS Attacks:
    - distinguishing between attack traffic and normal traffic is challenging, especially in the case of botnets performing HTTP Flood attacks.
    - each bot in a botnet makes seemingly legitimate network requests, making the traffic appear "normal" in origin.

  • Solution?
    - tools like a properly configured Web Application Firewall (WAF), are necessary for mitigation(ModSecurity).
    - filter traffic through an IP reputation database. LIKE THIS

  • Could not talk to a EBSI pilot server - need a "Verifiable Authorisation To Onboard"

  • Found and compiled the DataVault app from old master thesis. Spend days fixing compatibility and version errors. Updated libraries to match the newer kotlin version => working on my phone(except the performance test).

@synctext
Copy link
Member Author

synctext commented Feb 19, 2024

  • 🎉 🎉 🎉
    • Awesome core thesis idea
    • filter traffic through an IP reputation database.
    • draft thesis title: "IP reputation protection for passport-grade EU SSI ID server" (todo: less abbreviation)
  • Related work
  • fundamental issue is that reputation is tied to your passport-grade ID versus IPv4 address
    • not really totally ambitious, just fundamentally fix The Internet 🙄
    • ideas:
      • for every successful login, leave IPv4 in table (Goodness table)
      • for all irrefutable attack patterns, leave IPv4 in table (Bad table)
      • Are you a bot? Solve this Captcha or call your local EU municipality
      • You are a bot, you now go straight to bot jail
      • For each /24, /16, and /8 keep a block reputation
        • Multiple public keys of logins (or hashed IDs)
        • Multiple irrefutable attack patterns
        • Mixed 😲 😮 😨
        • Keep this smaller then complete BGP routing table (AS count 74,200)
  • We completely safe, except for "server-side request forgery" attacks.
  • Upcoming sprint. Read related work:
  • Future: top-down analysis, fixes problem?

@AdiDumi
Copy link

AdiDumi commented Mar 7, 2024

Sprint update:

  • not as much progress as hoped due to house hunting and stomach flu
  • studied more the ModSecurity with Core Rule Set:
    - combination of anomaly scoring = each rule that matches a part of the request contributes to a cumulative score. The more rules a request triggers, the higher its overall score becomes until a threshold set by user
    - and paranoia level the sensitivity or aggressiveness of the rule set, depending on the type of data accessed, problems with false-positives
  • working on docker files to have containers running the ModSecurity + SlowHttpTest with docker compose:
    - SlowHttpTest
    - ModSecurity
  • Changed dockerfiles from repos, work in progress to make them build, had to add versions, delete packages..
  • Paper shows that ModSecurity is vulnerable especially to DoS attacks(multiple types: slow header, slow body, range attacks)
  • ModSecurity has been passed even from 2014, presented in Analysis of Slow Read DoS Attack and Countermeasures
Full draft docker magic

ARG HTTPD_VERSION="1"

FROM httpd:${HTTPD_VERSION} as build

ARG MODSEC2_VERSION="1"
ARG LUA_VERSION="1"

RUN set -eux; \
    echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections; \
    apt-get update -qq; \
    apt-get install -y -qq --no-install-recommends --no-install-suggests \
        automake \
        ca-certificates \
        g++ \
        git \
        libapr1-dev \
        libaprutil1-dev \
        libcurl4-gnutls-dev \
        libfuzzy-dev \
        libpcre3-dev \
        libtool \
        libxml2-dev \
        libyajl-dev \
        lua${LUA_VERSION}-dev \
        make \
        pkgconf \
        wget

RUN set -eux; \
    wget --quiet https://github.com/owasp-modsecurity/ModSecurity/archive/refs/tags/v${MODSEC2_VERSION}.tar.gz; \
    tar -zxvf v${MODSEC2_VERSION}.tar.gz; \
    cd ModSecurity-${MODSEC2_VERSION}; \
    ./autogen.sh; \
    ./configure --with-yajl --with-ssdeep; \
    make; \
    make install; \
    make clean

FROM httpd:${HTTPD_VERSION} as crs_release

ARG CRS_RELEASE

# hadolint ignore=DL3008,SC2016
RUN set -eux; \
    apt-get update; \
    apt-get -y install --no-install-recommends \
      ca-certificates \
      curl \
      gnupg; \
    mkdir /opt/owasp-crs; \
    curl -SL https://github.com/coreruleset/coreruleset/archive/v${CRS_RELEASE}.tar.gz -o v${CRS_RELEASE}.tar.gz; \
    curl -SL https://github.com/coreruleset/coreruleset/releases/download/v${CRS_RELEASE}/coreruleset-${CRS_RELEASE}.tar.gz.asc -o coreruleset-${CRS_RELEASE}.tar.gz.asc; \
    gpg --fetch-key https://coreruleset.org/security.asc; \
    gpg --verify coreruleset-${CRS_RELEASE}.tar.gz.asc v${CRS_RELEASE}.tar.gz; \
    tar -zxf v${CRS_RELEASE}.tar.gz --strip-components=1 -C /opt/owasp-crs; \
    rm -f v${CRS_RELEASE}.tar.gz coreruleset-${CRS_RELEASE}.tar.gz.asc; \
    mv -v /opt/owasp-crs/crs-setup.conf.example /opt/owasp-crs/crs-setup.conf

FROM httpd:${HTTPD_VERSION}

ARG MODSEC2_VERSION
ARG LUA_VERSION
ARG LUA_MODULES

ENV APACHE_ALWAYS_TLS_REDIRECT=off \
    APACHE_LOGFORMAT='"%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""' \
    APACHE_ERRORLOG_FORMAT='"[%{u}t] [%-m:%l] [pid %P:tid %T] %7F: %E: [client\ %a] %M% ,\ referer\ %{Referer}i"' \
    APACHE_METRICS_LOGFORMAT='"%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""' \
    ACCESSLOG=/var/log/apache2/access.log \
    BACKEND=http://localhost:80 \
    BACKEND_WS=ws://localhost:8080 \
    ERRORLOG='/proc/self/fd/2' \
    H2_PROTOCOLS='h2 http/1.1' \
    LOGLEVEL=warn \
    METRICS_ALLOW_FROM='127.0.0.0/255.0.0.0 ::1/128' \
    METRICS_DENY_FROM='All' \
    MUTEX='default' \
    METRICSLOG='/dev/null' \
    MODSEC_AUDIT_ENGINE="RelevantOnly" \
    MODSEC_AUDIT_LOG_FORMAT=JSON \
    MODSEC_AUDIT_LOG_TYPE=Serial \
    MODSEC_AUDIT_LOG=/dev/stdout \
    MODSEC_AUDIT_LOG_PARTS='ABIJDEFHZ' \
    MODSEC_AUDIT_STORAGE=/var/log/modsecurity/audit/ \
    MODSEC_DATA_DIR=/tmp/modsecurity/data \
    MODSEC_DEBUG_LOG=/dev/null \
    MODSEC_DEBUG_LOGLEVEL=0 \
    MODSEC_DEFAULT_PHASE1_ACTION="phase:1,pass,log,tag:'\${MODSEC_TAG}'" \
    MODSEC_DEFAULT_PHASE2_ACTION="phase:2,pass,log,tag:'\${MODSEC_TAG}'" \
    MODSEC_DISABLE_BACKEND_COMPRESSION="On" \
    MODSEC_PCRE_MATCH_LIMIT_RECURSION=100000 \
    MODSEC_PCRE_MATCH_LIMIT=100000 \
    MODSEC_REQ_BODY_ACCESS=on \
    MODSEC_REQ_BODY_LIMIT=13107200 \
    MODSEC_REQ_BODY_LIMIT_ACTION="Reject" \
    MODSEC_REQ_BODY_JSON_DEPTH_LIMIT=512 \
    MODSEC_REQ_BODY_NOFILES_LIMIT=131072 \
    MODSEC_RESP_BODY_ACCESS=on \
    MODSEC_RESP_BODY_LIMIT=1048576 \
    MODSEC_RESP_BODY_LIMIT_ACTION="ProcessPartial" \
    MODSEC_RESP_BODY_MIMETYPE="text/plain text/html text/xml" \
    MODSEC_RULE_ENGINE=on \
    MODSEC_SERVER_SIGNATURE="Apache" \
    MODSEC_STATUS_ENGINE="Off" \
    MODSEC_TAG=modsecurity \
    MODSEC_TMP_DIR=/tmp/modsecurity/tmp \
    MODSEC_TMP_SAVE_UPLOADED_FILES="on" \
    MODSEC_UPLOAD_DIR=/tmp/modsecurity/upload \
    PORT=80 \
    PROXY_ERROR_OVERRIDE=on \
    PROXY_PRESERVE_HOST=on \
    PROXY_SSL=off \
    PROXY_SSL_CA_CERT=/etc/ssl/certs/ca-certificates.crt \
    PROXY_SSL_CERT=/usr/local/apache2/conf/proxy.crt \
    PROXY_SSL_CERT_KEY=/usr/local/apache2/conf/proxy.key \
    PROXY_SSL_CHECK_PEER_NAME=off \
    PROXY_SSL_CIPHERS="ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384" \
    PROXY_SSL_PROTOCOLS="all -SSLv3 -TLSv1 -TLSv1.1" \
    PROXY_SSL_VERIFY=none \
    PROXY_TIMEOUT=60 \
    REMOTEIP_INT_PROXY='10.1.0.0/16' \
    REQ_HEADER_FORWARDED_PROTO='https' \
    SERVER_ADMIN=root@localhost \
    SERVER_NAME=localhost \
    SERVER_SIGNATURE=Off \
    SERVER_TOKENS=Full \
    SSL_CERT=/usr/local/apache2/conf/server.crt \
    SSL_CERT_KEY=/usr/local/apache2/conf/server.key \
    SSL_CIPHERS="ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384" \
    SSL_ENGINE=on \
    SSL_HONOR_CIPHER_ORDER=off \
    SSL_PORT=443 \
    SSL_PROTOCOLS="all -SSLv3 -TLSv1 -TLSv1.1" \
    SSL_SESSION_TICKETS=off \
    SSL_OCSP_STAPLING=On \
    TIMEOUT=60 \
    WORKER_CONNECTIONS=400 \
    # CRS specific variables
    PARANOIA=1 \
    ANOMALY_INBOUND=5 \
    ANOMALY_OUTBOUND=4 \
    BLOCKING_PARANOIA=1

COPY --from=build /usr/local/apache2/modules/mod_security2.so                  /usr/local/apache2/modules/mod_security2.so
COPY --from=build /usr/local/apache2/ModSecurity-${MODSEC2_VERSION}/modsecurity.conf-recommended  /etc/modsecurity.d/modsecurity.conf
COPY --from=build /usr/local/apache2/ModSecurity-${MODSEC2_VERSION}/unicode.mapping               /etc/modsecurity.d/unicode.mapping
COPY --from=crs_release /opt/owasp-crs /opt/owasp-crs
COPY src/etc/modsecurity.d/*.conf /etc/modsecurity.d/
COPY src/bin/* /usr/local/bin/
COPY apache/conf/extra/*.conf /usr/local/apache2/conf/extra/
COPY src/etc/modsecurity.d/*.conf /etc/modsecurity.d/
COPY src/opt/modsecurity/activate-*.sh /opt/modsecurity/
COPY apache/docker-entrypoint.sh /

RUN set -eux; \
    echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections; \
    apt-get update -qq; \
    apt-get install -qq -y --no-install-recommends --no-install-suggests \
        ca-certificates \
        curl \
        gnupg \
        iproute2 \
        libcurl3-gnutls \
        libfuzzy2 \
        liblua${LUA_VERSION} \
        ${LUA_MODULES} \
        libxml2 \
        libyajl2; \
    update-ca-certificates -f; \
    apt-get clean; \
    rm -rf /var/lib/apt/lists/*

RUN set -eux; \
    mkdir -p /etc/modsecurity.d/; \
    mkdir -p /tmp/modsecurity/data; \
    mkdir -p /tmp/modsecurity/upload; \
    mkdir -p /tmp/modsecurity/tmp; \
    chown -R $(awk '/^User/ { print $2;}' /usr/local/apache2/conf/httpd.conf) /tmp/modsecurity; \
    mkdir -p /var/log/apache2/; \
    ln -s /opt/owasp-crs /etc/modsecurity.d/; \
    sed -i -E 's|(Listen) [0-9]+|\1 ${PORT}|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|(ServerTokens) Full|\1 ${SERVER_TOKENS}|' /usr/local/apache2/conf/extra/httpd-default.conf; \
    sed -i -E 's|(ServerSignature) Off|\1 ${SERVER_SIGNATURE}|' /usr/local/apache2/conf/extra/httpd-default.conf; \
    sed -i -E 's|#(ServerName) www.example.com:80|\1 ${SERVER_NAME}|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|(ServerAdmin) [email protected]|\1 ${SERVER_ADMIN}|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|^(\s*CustomLog)(\s+\S+)+|\1 ${ACCESSLOG} modsec "env=!nologging"|g' /usr/local/apache2/conf/httpd.conf; \
	sed -i -E 's|^(\s*ErrorLog)\s+\S+|\1 ${ERRORLOG}|g' /usr/local/apache2/conf/httpd.conf; \
	sed -i -E 's|^(\s*TransferLog)\s+\S+|\1 ${ACCESSLOG}|g' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule unique_id_module)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule rewrite_module modules/mod_rewrite.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule proxy_module modules/mod_proxy.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule proxy_http_module modules/mod_proxy_http.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule remoteip_module modules/mod_remoteip.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule socache_shmcb_module modules/mod_socache_shmcb.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule ssl_module modules/mod_ssl.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(LoadModule http2_module modules/mod_http2.so)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(Include conf/extra/httpd-default.conf)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(Include conf/extra/httpd-proxy.conf)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(Include conf/extra/httpd-ssl.conf)|\1|' /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|#(Include conf/extra/httpd-vhosts.conf)|\1|' /usr/local/apache2/conf/httpd.conf; \
    echo 'Include conf/extra/httpd-locations.conf' >> /usr/local/apache2/conf/httpd.conf; \
    echo 'Include conf/extra/httpd-modsecurity.conf' >> /usr/local/apache2/conf/httpd.conf; \
    sed -i -E 's|(MaxRequestWorkers[ ]*)[0-9]*|\1${WORKER_CONNECTIONS}|' /usr/local/apache2/conf/extra/httpd-mpm.conf; \
    chgrp -R 0 /var/log/ /usr/local/apache2/; \
    chmod -R g=u /var/log/ /usr/local/apache2/

ENTRYPOINT ["/docker-entrypoint.sh"]

FROM alpine:3.17 as builder

RUN apk add --no-cache build-base git openssl-dev autoconf automake
WORKDIR /build
COPY . /build
RUN ./configure && make


FROM alpine:3.17
RUN apk add --no-cache libstdc++
COPY --from=builder /build/src/slowhttptest /usr/local/bin/
ENTRYPOINT ["slowhttptest"]

services:
  slowhttp:
    build:
      context: ./slowhttp

  modsecurity:
    build:
      context: ./modsecurity

@synctext synctext changed the title msc placeholder: PKI, EBSI, and SSI EBSI digital identity DDoS hardening using IP reputation Mar 8, 2024
@synctext
Copy link
Member Author

synctext commented Mar 8, 2024

@AdiDumi
Copy link

AdiDumi commented Mar 26, 2024

Update sprint:

  • ModSecurity can be deployed as a reverse proxy for the server so the traffic is not encrypted

image

  • by default, ModSecurity operates at the application layer and is not designed to directly manage login sessions or track successful login attempts. But, it can be configured to work in conjunction with other security mechanisms, such as rate limiting or custom rules, to detect and mitigate potential threats, including brute-force login attacks:
    - Rate Limiting: implement rate limiting rules that throttle the number of login attempts from specific IP netranges. This can help mitigate brute-force attacks by limiting the number of requests allowed within a certain time frame.
    - Custom Rules: custom ModSecurity rules to detect and block login attempts from specific IP netranges. These rules could analyze patterns in the request headers or parameters to identify login attempts and then take appropriate action based on the originating IP address.
  • slowhttptest container build and run ✔️
  • modsecurity container build and run ✔️
  • still problems in connecting the containers but docker-compose and Dockerfile clean up, short and simple commands
  • update - connection established between containers ✔️
  • working in repo

image

@synctext
Copy link
Member Author

synctext commented Mar 27, 2024

@AdiDumi
Copy link

AdiDumi commented Apr 14, 2024

Short sprint update:

Problem_Description_thesis (1).pdf

@synctext
Copy link
Member Author

synctext commented Apr 15, 2024

  • as usual, solid progress 🚀
    • start to look like a thesis
    • Only 5-6 experimental result figures, more text, DONE!
  • Good related work section
  • Fundamental security problem with Internet architecture (e.g. Problem Description)
  • Sprint proposal:
    • given IP reputation focus, IETF Dots, Trustguard, and Modsecurity
    • come up with 3 thesis direction for implementation and performance analysis 😄 Easy sprint
    • plus needs to evolve in a generic Fix-The-Internet 👍
    • {idea III: login-once, light-weight token check for every API call, large-scale zero-trust tracking {epic table of reputations with O(1) in 1 TByte}} { ⚡ fails to use coordination by network intermediaries}
    • Zero trust is based on the principle of “never trust, always verify.”

@AdiDumi
Copy link

AdiDumi commented Apr 30, 2024

Brainstorming sprint:

  • ideas for directions:
    1. Design smart contracts to manage reputation scores and updates on the blockchain. Each reputation update (e.g., positive or negative feedback) is recorded as a transaction on the blockchain, ensuring transparency and immutability. Extend ModSecurity to interact with the blockchain for reputation validation. When processing incoming requests, ModSecurity queries the blockchain to verify the reputation score of the requesting IP address and incorporate a "TrustGuard's like" reputation management techniques to dynamically adjust reputation scores based on network interactions.
    2. Deploy the IETF DOTS protocol with blockchain-based IP reputation management for DDoS mitigation in distributed server environments, real-time threat signaling and coordination during attacks. Manage IP reputation data securely across distributed DOTS agents and servers. Configure DOTS agents to interact with the blockchain for reputation validation and retrieval during DDoS attack mitigation
    3. Zero-Trust Reputation Management for API security, login-once mechanism for API -> authentication tokens -> lightweight token checks into API endpoints to validate the authenticity and permissions of incoming requests -> IP reputation check
    4. Simple reputation scores and updates on the blockchain for every IP or range/16. Define a harsh reputation computation if heavy traffic comes from IP, users usually have something that DDos attackers do not have -> time to get back the reputation -> time for the server to respond to an attack. The important part: to define a good calculation of reputation based on factors like nr request/second + history over time + successful accesses + bad requests etc.
  • performance analysis:
    - blockchain transaction throughput
    - overall attack detection rate
    - mitigation success rate
    - system overhead during attack
    - attack detection accuracy
    - response time for blocking malicious traffic

@synctext
Copy link
Member Author

synctext commented May 1, 2024

😲 😲 😲 You have a viable roadmap to fix The Internet.

P2P Modsecurity is a brilliant idea. With a bit of tweaking it is also incrementally expandable. However, no smart contracts, gas payments, and Turing incompleteness please.

Re-architecting The Internet using zero-trust principle, Modsecurity, and PUFs

We create a layer of trust The Internet always needed, but never had. In 1962 the architecture of The Internet for the thermonuclear era was written down in report P2626. The highly survivable system structure is fundamentally unsuitable for today's world. For instance, Internet address 180.101.88.232 owned by ISP ChinaNet Jiangsu Province has been launching SSH login attacks for multiple years.
First we create a universal trust token. It consists of a non-revocable self-sovereign identity with list of trust attestations. Second, we instrument Modsecurity with trust scoring, real-time threat signalling, coordination with others, and automatic formation of a global web-of-trust. Third, we present a trust model which is grounded in the laws of physics and mathematical axioms. By combing zero-trust principle and physical unclonable functions we create strong identity and web-of-trust framework which can serve as a mid-life upgrade of The Internet.

Roadmap till 31 Oct graduation

  • priority is realisation of zero-trust architecture. All performance or specific features are secondary. Proof-of-principle running code.
  • Modsecurity IPv4 scoring for just two trivial attack types (SSH login FAIL, SlowHHTPtest).
    • exchange of universal mistrust tokens about attacks
    • both IPv4 mistrust or self-sovereign identity abuse.
    • web-of-trust creation
    • re-usage of either our mature codebase of Python-based IPv8, trustchain, and EVA. Or compiled Kotlin, trustchain, and uTP.
  • Modsecurity understanding of the universal trust token
    • add trust attestations to universal trust token
    • trust anchors can sign the self-sovereign ID.
  • PUF part: only in text and pictures of prior work, no coding
  • classical thesis layout: problem, design, implementation and performance analysis
    • first experiment: server hammering. modsecurity login fails. show creation of mistrust records. Repeated offenders get blocked faster using web-of-trust.
    • second experiment: SlowHHTPtest
    • third experiment: SSI wallet collecting trust signatures
    • final experiment: Attack of the stolen wallets. numerous wallet abusive repeated logins, real-time threat signalling, web-of-trust prevents repeated attacks on other Modsecurity instances

update: strong related work of a -simulation- of attack info sharing https://github.com/LukasForst/fides + proper code: https://github.com/stratosphereips/StratosphereLinuxIPS

@AdiDumi
Copy link

AdiDumi commented May 22, 2024

Back to coding 🎊
The first experiment work in progress. https://github.com/AdiDumi/IpRepMaster
Created a simple login python app to protect and containerized it with Apache2 ModSecurity which acts as a proxy for requests and responses. Every login fail is detected by the modsecurity in the response.
image
Clean up on the last docker container used. Made it more simple and efficient (building was taking too long). -> very basic configuration with logging enabled for detection rule.

# Basic configuration
SecRuleEngine On
SecRequestBodyAccess On
SecResponseBodyAccess On
SecStatusEngine Off

# Enable audit logging
SecAuditEngine On
SecAuditLogType Serial
SecAuditLogFormat JSON
SecAuditLogParts ABIJDEFHZ
SecAuditLog /var/log/apache2/modsec_audit.log

Created custom rules for modsecurity to apply on failed login and detect. Lots of options from documentation with different level of details to add to the logging and rules for specific requests https://github.com/owasp-modsecurity/ModSecurity/wiki/Reference-Manual-%28v3.x%29

# Log failed login attempts
SecRule REQUEST_URI "/login" "phase:1,log,auditlog,id:1001,msg:'Login attempt detected'"
SecRule RESPONSE_STATUS "@streq 403" "phase:3,log,auditlog,msg:'Failed login attempt detected',id:101"
Logging in an audit.log file that creates JSON for a transaction
{
  "transaction": {
    "time": "20/May/2024:09:29:53.332742 +0000",
    "transaction_id": "Zk27Eeazul3OewgjLhj2PAAAAAA",
    "remote_address": "172.17.0.1",
    "remote_port": 37570,
    "local_address": "172.17.0.2",
    "local_port": 80
  },
  "request": {
    "request_line": "POST /login HTTP/1.1",
    "headers": {
      "Host": "localhost",
      "User-Agent": "python-requests/2.32.0",
      "Accept-Encoding": "gzip, deflate",
      "Accept": "*/*",
      "Connection": "keep-alive",
      "Content-Length": "35",
      "Content-Type": "application/x-www-form-urlencoded"
    },
    "body": [
      "username=admin&password=faspassword"
    ]
  },
  "response": {
    "protocol": "HTTP/1.1",
    "status": 403,
    "headers": {
      "Content-Type": "application/json",
      "Content-Length": "27",
      "Keep-Alive": "timeout=5, max=100",
      "Connection": "Keep-Alive"
    },
    "body": ""
  },
  "audit_data": {
    "messages": [
      "Warning. String match \"403\" at RESPONSE_STATUS. [file \"/etc/modsecurity/custom_rules.conf\"] [line \"3\"] [id \"1001\"] [msg \"Failed login attempt detected\"]"
    ],
    "error_messages": [
      "[file \"apache2_util.c\"] [line 275] [level 3] [client 172.17.0.1] ModSecurity: Warning. String match \"403\" at RESPONSE_STATUS. [file \"/etc/modsecurity/custom_rules.conf\"] [line \"3\"] [id \"1001\"] [msg \"Failed login attempt detected\"] [hostname \"localhost\"] [uri \"/login\"] [unique_id \"Zk27Eeazul3OewgjLhj2PAAAAAA\"]"
    ],
    "handler": "proxy-server",
    "stopwatch": {
      "p1": 589,
      "p2": 882,
      "p3": 75,
      "p4": 115,
      "p5": 1215,
      "sr": 111,
      "sw": 1115,
      "l": 0,
      "gc": 0
    },
    "response_body_dechunked": true,
    "producer": [
      "ModSecurity for Apache/2.9.7 (http://www.modsecurity.org/)",
      "OWASP_CRS/3.3.4"
    ],
    "server": "Apache/2.4.59 (Debian)",
    "engine_mode": "ENABLED"
  }
}
Working on processing the JSON transaction through a script that runs every time the log file is updates with a new entry(crons). Extract the relevant information about the login fail. Example:
{
  "ip_address": "172.18.0.1",
  "request_method": "POST",
  "request_uri": "/login HTTP/1.1",
  "status_code": 403,
  "message": "Warning. String match \"403\" at RESPONSE_STATUS. [file \"/etc/modsecurity/custom_rules.conf\"] [line \"3\"] [id \"1001\"] [msg \"Failed login attempt detected\"]"
}

Next steps:

  • creating rules to block the traffic after some failed attempts
  • understand and add the necessary layers and rules from OWASP Core Rule Set (CRS) (really big, not all needed probably)
  • connect the ModSecurity servers to a web-of-trust -> IPv8 python?

@synctext
Copy link
Member Author

synctext commented May 22, 2024

  • Solid progress! Exploring which idea is realistic for a thesis
  • Is connecting ModSecurity servers a good idea?
    • enable spreading an attack alert
    • negative feedback to N servers is an attack vector!
  • Novel zero-trust architecture
    • blacklisting based on failed logins and spam traffic
    • whitelisting based on trust token
    • {wild idea of the month, speculative research}
      • portable passport-grade trust for 5G
      • trust token in P2P mode for wallet-to-wallet communication
      • using IPv8 and phone-to-phone communication (e.g. NAT puncturing)
      • build a web-of-trust using signatures and blocks.
  • high-risk sprint focus: trust token
    • assume public-key crypto
    • login to server, server signs certificate
    • bundle of certificates parsed by ModSecurity
    • becomes a trust token 💥

@AdiDumi
Copy link

AdiDumi commented Jun 11, 2024

Work in progress:

  • added openssl private/public key creation for each server
  • now I can create multiple docker containers/servers each one with its own key pair -> simulation of distributed servers
  • each server has also a unique ID to use in the token
  • working on each server adding in the response a signed token with its ID
    private_key.sign( certificate.encode(), padding.PSS( mgf=padding.MGF1(hashes.SHA256()), salt_length=padding.PSS.MAX_LENGTH ), hashes.SHA256() )
  • each server checks the token received with a request and checks the integrity of the certificates(by other servers)
    public_key.verify( signature, certificate.encode(), padding.PSS( mgf=padding.MGF1(hashes.SHA256()), salt_length=padding.PSS.MAX_LENGTH ), hashes.SHA256() )
  • ModSecurity can call this scripts and decide what to do with the request(drop, forward, more restrictive rules etc)
  • using cryptography python package
    Code for this stage of development (with Dockerfile containing creation of key-pair) repo

@synctext
Copy link
Member Author

synctext commented Jun 11, 2024

🎉 🎉 🎉

  • now time to add more hype (/s) either Edge-AI or zk-snarks. Again, you only need to invent a public trust function to save The Internet...
  • Self-sovereign, zero-trust architecture. So users store all the trust tokens. No 'helpfull' Big Tech cloud storage.
  • Can ModSecurity do high-performance trust parsing? Function calling, interrupts, stack duplication, cache flush for each UDP packet?
  • Fundamental design choice: transparent trust communication or explicit trust protocol
    • eat away the magic trust token. do not interfere with any server protocol, just add another header session management logic, encapsulation. Basically re-building QUIC Permanent Connection ID for trust. Or SIP for trust.
    • passively observe if trust exists and signature is valid; then forward to server.
    • Both? Explicit forwarding server (trust protocol) or transparent DDoS filter (drop or forward)
  • Running the ZK-SNARK proving algorithm for 448 million people of the EU
  • SDSI Reborn???
    • Establishing Identity Without Certification Authorities
    • Our only purpose is to bind several signature to a public key, in order to prove prior existence. No identity needs to be established. Such proof-of-prior-interactions is not a novel concept. It just failed to materialise for 28 years 🙃
    • re-pimp The Ancient Texts: RFC2692,RFC2693
    • Every server becomes a self-certifying peer. Talk to many peers, collects signatures, build trust, Re-decentralise The Internet. Essential: score the signatures into a trust estimation. Sybil attack, free identity creation
    • great idea, because others also are working on it: called Rate Limiting Nullifier. (idea: make it simple. Follow this approach exclusively with tech that is at least 30-years old; serialised signatures). Swapping signatures with others to enhance privacy is allowed!
  • Final thesis experiment? Do this with upcoming EBSI EU digital identity: score of 100% passport-grade trust.
  • Be goal-oriented towards EBSI Grand Experiment. So no 10-week expiration token refresh code please

Next sprints: idea gets bigger, experiments become more focused. 15Sep talk to EBSI server or cancel that part.

@AdiDumi
Copy link

AdiDumi commented Jun 28, 2024

Experiment update:

  • creating the full extension rule set for checking the signatures of a public key and getting a score
  • for now in each request there is a public key header and a signature header with the list of servers that signed before
    [ { "id": 1, "signature": "abc123" }, { "id": 2, "signature": "def456" }, { "id": 3, "signature": "ghi789" } ]
  • scripts that check the signatures and computes a score -> 1 signature is wrong? deny, no signature? more strict rules to apply(TODO)
  • on a successful response append the server signature to the list and give it back to the user(full circle). The list of signatures is a set(no double check for the same server)
  • simple crypto for signing and checking
  • still some problems in parsing the signatures in ModSecurity with scripts call in python
  • still open issues from ModSecurity that I have to work around it(https://github.com/owasp-modsecurity/ModSecurity/issu es/2167)
  • until now, no related work of ModSecurity and cryptography, novel?
  • problems with parameters parsing in exec function -> need a workaround for a single parameter or intermediate log
Full Rules set
SecRule REQUEST_URI "/login" "phase:1,log,auditlog,id:1001,msg:'Login attempt detected'"
SecRule RESPONSE_STATUS "@streq 403" "phase:3,log,auditlog,msg:'Failed login attempt detected',id:101"
SecRule RESPONSE_STATUS "^200$" \
    "id:10003, \
    phase:4, \
    t:none, \
    pass"

SecRule RESPONSE_HEADERS:X-User-Public-Key "!@streq 0" \
    "id:10008, \
    phase:4, \
    t:none, \
    pass, \
    setvar:tx.header_value=%{RESPONSE_HEADERS.X-User-Public-Key}, \
    log, \
    msg:'received message', \
    exec:'/app/add_token.py %{tx.header_value}'"

SecRule REQUEST_HEADERS:X-User-Public-Key "!@streq 0" \
     "id:4001, \
     phase:1, \
     t:none, \
     setenv:key=%{REQUEST_HEADERS:X-User-Public-Key}, \
     deny, \
     log, \
     msg:'Missing X-User-Public-Key header'"

SecRule REQUEST_HEADERS:User-Signatures "!@streq 0" \
    "id:4002, \
     phase:1, \
     t:none, \
     setenv:sign=%{REQUEST_HEADERS:User-Signatures}, \
     pass, \
     nolog"

SecRule REQUEST_HEADERS:User-Signatures "!@streq 0" \
    "id:4009,phase:1,t:none,exec:'/app/check_signatures.py %{env.sign} %{env.key}',setenv:EXEC_RESULT=%{TX.exec.ret}"

SecRule ENV:EXEC_RESULT "@eq 0" \
    "id:7002, \
     phase:1, \
     t:none, \
     deny, \
     log, \
     msg:'Python script denied the request'"

SecRule ENV:EXEC_RESULT "!@eq 0" \
    "id:7003, \
     phase:1, \
     t:none, \
     pass, \
     log, \
     msg:'Python script allowed the request'"

@synctext
Copy link
Member Author

synctext commented Jun 28, 2024

  • Good point! Your design requires a "connection state table" with each known public key (origin ip:port uniqueness!)
  • Server needs to tell modsecurity or other shell utility about successful enrolments and logins
  • Protocol agnostic: fail to login, maximum re-tries reached, excessive requests--go away,and welcome message is indistinguishable for modsecurity
  • Then we need the concept of sessions and timeouts for stale entries in "connection state table" 😨
  • DDoS modsecurity with state explosion attack 💀 ☠️ 💀
  • Background note: passport-grade identity infrastructure is -very- costly
  • we're adding a new Internet layer: trust
    • current execution environment: modsecurity
    • DDoS as a first focus, 1st line of defence (can be leaky, stochastic)

ToDo: write small text with state-of-the-art in DDoS + IPv4 reputation. Expand experiment.

@AdiDumi
Copy link

AdiDumi commented Sep 2, 2024

Back to school with new experiment results.
First some bad news:

  • this summer from 1st July ModSecurity is announced as End of Life for commercial support with the main team maintaining stepping out and ending completely support for nginx servers!! news
  • rules that call external scripts can not give arguments(the parsing is static so no dynamic values from request etc)
  • parsing can be done dynamically only with LUA scripts, not really supporting python
  • rollercoaster of emotions: found a library(libmodsecurity) by modsec team that supports other languages(python, C etc) 🥳 -> after experiments with it found out it did not work even for them and stopped it, even having a label with 😥 "RIP - libmodsecurity" - realeased in 2018 - died in 2018 ⚰️
  • modsecurity can not alter the response headers without being disruptive(stopping the response and reconstructing it)

BUT there are good news for them:

  • Basic prototype working of a trust token generation, parsing and scoring with basic cryptographic operations between servers in python
  • Mod_Security still maintained by the community now in the hands of OWASP! already re-opened the website maybe more support coming
  • the experiment is using apache2 not nginx so nothing to worry about the server
  • tried every rule and found a workaround calling python scripts with the rule "@inspectFIle" 😎, but it can only take 1 argument from the request so I aggregated them in a header
  • for now the server creates the response header by adding the signed public key of the user(the only requirement for the server)
    About the experiment:
  • Added even more rules that now can block requests with wrong signatures -> log the ip of the respectiv wrong request
  • Computing and storing a score for the request = how many good signatures has from other servers per public key
    { "LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF0UXJYeHYxaUJLTUNXZDB4WStRUwpZU0NoY20zcXczbU1MZVV1end6KzBvUUljUjR5YkFoY1Joa3c3NDAyQ1lDTTZObDYvZzRzOEZ2VlhwUXUyZ3F5CnJKcDhobkZ3THhjTDlDZ25KWWtzR3BEazFMTUZMWWE4WUhYQUhiVjZpVnlKM3FZSnRyUXVhd2YwWGpQd21jVEwKaGVYcG1UL1o1elQ4R0JON0xJN2hOc1gxT0w0Qk1HMzI2clVWN3NHbjRNMEdvWXdQN1pRWHAzOHMveEZjYXo2QgozWE16WjVsaFViSjY1NkJSVEVCcFpFeUE5UzUra1NlN3YrbTB6T3ZkTFhMREo3NTVMUTBleUJBbTJuRGw0TVFrCmNtZHhFNmUvUkFXUEVuVjROaUlYSUlKQTZsOVVqNG1lL3RUMDN2Q1hzTGFnblVIK3owT2RNeW5zb3dOSHM3Vk4KRndJREFRQUIKLS0tLS1FTkQgUFVCTElDIEtFWS0tLS0tCg==": 2 }
  • created a record for ips blocked because of signatures
    [ "172.17.0.1" ]
  • if you are trying to temper with the signatures or bad headers - blocked 🔨
<title>403 Forbidden</title>
  • Mod Security has rules for IP whitelisting or blacklisting based on txt file of address!! or even from another server that maintains the lists -> this can be very helpful
    SecRule REMOTE_ADDR "@ipMatchFromFile ips.txt" "id:163"
    system drawio
  • there is no prior project/state-of-the-art that combines Mod Security with cryptography or IP reputation

Code with the latest experiment

Next steps perhaps:

  • log good ips and somwhow make a connection between public key score and ip(combine public key + ip as a identifier)
  • deploy experiment with multiple servers
  • distribute this whitelist - blacklist of ips between servers, distributed database?
  • decide on how to use the lists: first check signatures then check the ip list to see if you check more? or only the login page checks for ips?

Description and related work.pdf

@synctext
Copy link
Member Author

synctext commented Sep 2, 2024

Solid progress. With the first operational trust token, now see how this would "sell" in a master thesis. DDoS is still the main attack. If you do the same work, but titled "Zero-Trust Architecture", then it is suddenly scientific 🤡 🧐 🤡

Linkage of zero-trust architecture to first principles of digital identity. Key thing is to make it as expensive as possible to setup, maintain, and/or exit fake identities. No focus on the detailed scoring, focus on the realisation of the overall architecture. What would 3 convincing zero-trust EBSI passport-grade ID experiments look like?

  • Analyse performance of processing a single trust token
  • Trust token with multiple signatures with differing associated levels of assurance (low,medium,high)
    • Milestone: devices carry identity management info and credentials from various trust domains
  • Trust tokens arriving at 100 per second
  • DDoS attack traffic mixed with high-trust tokens
  • Trust tokens generated by both servers and other citizens (transitive trust). The EBSI server gives the citizen the ability to issue trust themselves. This gives them a high level of autonomy and ability to construct a trust network. This goes beyond the mere delegation of trust from at most 1 identity to an object. With transitive trust multiple identities attest the trustworthiness of a single receiver. This receiver can be another person, organisation, car/ship, or object in general.
    • Transitive trust in mobile scenarios
    • great critical paper: Zero Trust in Zero Trust
    • Assuring the Safety of Asymmetric Social Protocols (Transcript of Discussion)
    • Trust refresh service (1 month old trust versus 1 second ago)
    • Transitive trust token in non-interactive zero-knowledge proof form
      • 1 new identity has zero trust
      • N high-trust identities, given by passport-grade EBSI identity service
      • N identities give a transitive trust token to newborn ID (endorsement operation)
      • N high-trust IDs go offline, also EBSI server goes offline
      • new identity can irrefutably prove it's trusted by N IDs.
        • zero privacy leakage
        • provably unique and real N IDs
        • No Sybil risk (or merely mention and leave for future thesis)
          • Compromised identities
          • Are endorsements unbounded?
          • Bounded transitive trust tokens per ID?
          • Do we need to restrict the outgoing trust "branches" ?
          • Each outgoing trust branch dilutes the amount of transitive trust it gives (SybilRES)

TrustZero: Passport-grade trust at Internet scale

Our audacious aim is to secure the European digital economy. We present the TrustZero architecture, a zero-trust architecture based upon the legal framework of European Union eIDAS for passport-grade digital identity. Specifically, TrustZero is meticulously design to provide passport-grade identity and trust for all financial transactions, information flows, and socio-economic activity in general. These four fundamental security assertions are provided by TrustZero:

  1. Presence of a live, non-cloned, and non-spoofed person. This can for instance be carried out by a selfie which is verified against photograph within the traditional paper-based passport. (holder binding)
  2. Integrity of the identity. This may be realised by a digital equivalent of a birth certificate, a legal document with the legal backing of correctness. (authentic source)
  3. Unaltered identity and unaltered identity attributes. Trusted cryptographic software must establish if any alteration have been made to identifiers or attributes such as name, place of birth or date of birth. (Person Identification Data)
  4. Integrity of the system and its components. Integrity assurance is based on independently-verifiable path from public source to code. For instance, using bit-by-bit identical reproducible builds facilitates verification that no vulnerabilities or backdoors exist. (end-to-end integrity assurance)

TrustZero thus first confirms who you are with absolute certainty. We deliberately selected the passport as the root-of-trust within TrustZero. Passports offer unrivalled security, broad usage, and deep historical grounding. Digital wallets on your smartphone are the modern equivalents of passports. {ToDo: 🚧 Byzantine robustness is critical for our envisioned societal primitive. TrustZero guarantees protection against any Byzantine communication failure, when the private keys remain private [REF]}.

Transitive trust and generic value transfer are the two main elements of TrustZero on top of identity management. It grants citizens autonomy, facilitates ad-hoc trusted collaboration, ability to construct trust networks, and establish trustworthy socio-economic activity in general. Our transitive trust and value transfer elements are designed to be long-enduring and evolvable, inspired by TCP/IP, Email, and HTTP. The TrustZero trust model goes beyond the mere delegation of trust from at most 1 identity to an entity. With transitive trust multiple identities attest the trustworthiness of a single receiver. This receiver can be another person, organisation, car/ship, object, or entity in general. TrustZero could potentially re-establish trust in social media, product review sites, and other Internet services. In prior work we designed and deployed a digital Euro with live connectivity to the legacy IBAN banking system of Europe. TrustZero digital wallets may contain fiat money, promissory notes, Bitcoins, or tokens of vaulted gold with outright ownership of certain physical gold. Previously we also designed and implemented an offline-first currency system by using Groth-Sahai proofs. Transactions can be executed offline without third parties and retroactive double-spending detection is facilitated.

Collective intelligence is a specific and powerful socio-economic activity enabled by TrustZero.
For instance, approaches such as Byzantine-robust learning with compression could be shielded by TrustZero from misinformation, deception, spam, and fraud. Exploring this direction is left as future work. {Todo2: Based on Trustzero we designed a DAO based on democratic goverance principles} {ToDo3: this work is based on 25 years of prior learn-by-doing work [ref1,ref2,bartercast,info_pools,tribler,SSI,SybilRes,distr-SGD...] within 5 project (19+7+4+2+.5MEuro...)}

No design chapter yet. For 1 sprint write a thesis-ready introduction+Problem Description.

@AdiDumi
Copy link

AdiDumi commented Sep 23, 2024

  • first measurement of the experiment with 5 servers running and users making correct/wrong request with signatures

experiment

  • expanded to more users making requests and storing up to 5 signatures(for now)
  • experimented with 1000 users sending requests from multiple threads -> ALL SERVERS RUNNING, no crashes
  • worked on the introduction and problem description
  • distancing from DDos mitigation and getting closer to a zero-trust architecture -> related work is still about the first one, changes to be made
  • inclusion of EBSI in thesis?
    Updated thesis.pdf

@synctext
Copy link
Member Author

synctext commented Sep 23, 2024

  • DDoS is indeed difficult as central storyline. Can't beat dedicated networking gear for that.
  • TrustZero: Passport-grade trust at Internet scale
    • Can this be tweaked into a real title?
    • Zero trust architecture realisation
    • Big Tech impact: EU law will mandate EBSI login. No need to create accounts anymore, you can login with EBSI account.
    • Benchmarking results
      • Server Modsecurity benchmark of trust token
      • same trust token between users Superapp Android
      • performance of 10k trust token database on Android (friend-of-friend-of-friend).
  • Re-architecting The Internet using zero-trust principle {repeating}
    We create a layer of trust The Internet always needed, but never had. In 1962 the architecture of The Internet for the thermonuclear era was written down in report P2626. The highly survivable system structure is fundamentally unsuitable for today's world. For instance, Internet address 180.101.88.232 owned by ISP ChinaNet Jiangsu Province has been launching SSH login attacks for multiple years.
    As a proof-of-principle we created a universal trust token. It consists of a non-revocable self-sovereign identity with list of trust attestations. Second, we instrument Modsecurity with trust scoring, real-time threat signalling, coordination with others, and automatic formation of a global web-of-trust. Third, we present a trust model which is grounded in the laws of physics and mathematical axioms. By combing zero-trust principle and physical unclonable functions we create strong identity and web-of-trust framework which can serve as a mid-life upgrade of The Internet.
  • Can we get pre-production EBSI to work with the data-sharing wallet?
  • we generate and release a large DNS features dataset of 400,000 benign and 13,011 malicious samples processed from a million benign and 51,453 known-malicious domains from publicly available datasets. The malicious samples span between three categories of spam, phishing, and malware., https://www.unb.ca/cic/datasets/dns-2021.html
  • Transitive trust dataset. Signed networks : networks with positive and negative edges (friend/foe, trust/distrust)
  • Current form of simple signature is too strong 👀 Signatures are very much the hammer of cryptographic primitives. As well as authenticating a message, they also provide third-party verifiability and (part of) non-repudiation.

ToDo: start writing first 4-5 pages of thesis!

@AdiDumi
Copy link

AdiDumi commented Oct 7, 2024

  • Writing spree ⌨️
  • updated thesis
    Thesis update.pdf
  • researched more about what we discussed, current signatures give out too much information about past interactions
  • group signatures might be a solution, you can check a signature in the group but still do not know who did it, no one know where you been before but they can verify and trust ⚡
  • deleted most of DDoS and EBSI references, still a good start for such an architecture

@synctext
Copy link
Member Author

synctext commented Oct 7, 2024

This sprint: get first experiments into thesis (text+graph). Who, Why, What + How

  • Trust token basics
  • signatures per second
  • 40,400, 400k domains in trust function.
    • many servers can issue trust tokens. hold in database and scoring function
    • No such thing as bad results, just show how/when it crashes.

@AdiDumi
Copy link

AdiDumi commented Oct 21, 2024

  • Creating more experiments
  • Managed to crash my own PC - out of CPU and memory while running experiment - corrupted files
  • Deploying 1000 users that send request continously - PC crashed but Modsecurity/server not - rate limiting very strong
    test2
  • at one point windows firewall kicks in with the docker and interrupts the connection -> not possible to send any more requests and only a full restart resets it. Gets hit by:
    ERROR: ('Connection aborted.', error(104, 'Connection reset by peer'))
  • Not even restarting docker solves it, windows probably just refuses to send any more requests
  • considering other type of signatures in terms of size
    testsize
  • polishing the text from protocol to implementation
  • because of PC crashes during experiments I was not able to experiment with OpenZiti
  • Progress now:
    Problem_Description_thesis (4).pdf

@synctext
Copy link
Member Author

synctext commented Oct 21, 2024

Great step forward in thesis 🎉

  • Most readable explanation of zero trust by Tailscale
  • Microsoft uses zero trust as foundation for Azure
  • Google got hacked and uses zero trust exclusively for their own security.
    • 2014 work, no real open source tech sharing. Just general architecture.
    • access depends solely on device and user credentials, regardless of a user’s network location—be it an enterprise location, a home network, or a hotel or coffee shop. All access to enterprise resources is fully authenticated, fully authorized, and fully encrypted based upon device state and user credentials.
  • Amazon has the most sweet one: Embracing Zero Trust: A strategy for secure and agile business transformation
  • Problem description: openness of best-in-class security which is used and kept secret by Big Tech and Big Banks.
    • significant resources to keep bad actors out
    • bad actors are getting smarter
    • zero-trust is the best defence we have.
      • Yet immature, requires combining various open source projects.
      • failure to form an open standard solution
      • we're left with "invent your own cryptography" at the corporate level
  • Design Principles section missing. Simply what are you doing: using weak login accounts at various services, each succesfully login creates a ZeroTrustToken, several tokens can be added to enhanced trust, multiple tokens enhance trust.

update:

Great related tooling survey in this paper. Re-use this info please.

Would advise you to delete Introduction and start over with different tone. Zero trust is not a hard-core systems topic. It's not about throughput, cache hits, tps, and latency. So outside our comfort zone. All this fluffy noisy info out there says its about consultancy fees, management buy-in, B2B sales pitches, awareness programs against an advanced threats, and getting hacked.

Introduction storyline...
The global competition for economic, military and technological dominance is altering the digital landscape. A global trade exists around security vulnerabilities in software for which no defence exists, called zero-days. Governments use zero days for military, intelligence, and law enforcement cyber operations, and criminal organizations use them to steal information and disrupt systems. REF

The zero trust movement is a reaction to this evolving cybersecurity armsrace. Simply using a VPN to encrypt traffic on The Internet is no longer sufficient in the age of zero-days trading. According to the European Systemic Risk Board report: The persistently heightened cyber threat environment in Europe is evidenced by the sabotage of undersea telecommunications cables, the disruptions to systems in large banks and third-party providers, and the increased sophistication of cyberattacks and the skills of hackers.
Attacks for pure financial gains in the financial sectors have evolved considerably in the past decades. In the TIBER-NL programme, financial companies test their resilience against advanced cyberattacks using test attacks that are based on realistic threats. The attacks are carried out by cybersecurity companies under the guidance of the Dutch Financial Market Authority REF. At the European level, a similar program exists operated by the European Central Bank REF

Big Tech giant Google was successfully "hacked" by China, as revealed in official company and US government publications. On January 14, 2010, the United States learned of a coordinated, sophisticated, and prolonged computer attack that targeted Google, Adobe and over 30 Fortune 100 companiesREF. Dubbed Operation Aurora after a folder found in an infected machine, the attack used a novel zero-day exploit. State-sponsored hackers focus on attacking and infiltrating second-tier defence industry suppliers that make electronic or mechanical components for major defence companies. Those companies then become a cyber "stepping stone" to gain access to the major defence contractors TEXT FULLY COPIED FROM WIKIPEDIA.

Problem Description
The zero trust movement lacks transparency, openness, standardisation, reproducibility, and knowledge exchange in general. No open source reference architecture and reference implementation exists which showcases best-in-class zero trust security. Reproducibility is lacking in the scientific literature about zero trust. The details of attacks, defensive software architectures, security software tooling, and security policies for cybersecurity remain secretive. This lack of openness even exists within academia. For instance, all offices at Delft University of computer science professors are public access, with the exception of cybersecurity.
With security based on the laws of physics, see low-cost HSM

After the successful attack on Google they migrated all their legal, financial, engineering, and administrative staff into isolated network domains using the zero trust principle. However, their description is very high level and lacks the details needed for scientific reproducibility.

design
We designed ZeroTrust based on a novel idea: strength through complexity. We use the inherent complexity to our advantage within our ZeroTrust framework. Complexity is often source of fragility, weakness, and faults in systems. For example, the Log4j vulnerability and crowdstrike global outage showed that global technology is based on shaky foundations REF. Complexity is the main challenge for defence contractors, airplane manufacturers, and the semiconductor industry.

Tech companies such as ASML have over 5000 suppliersREF. We use this level of complexity, depth, and diversity of the modern supply chain to our advantage. All prior work operated zero trust from the perspective of a single business entity, isolated organisation, or solitary government. ZeroTrust is based on the novel principle of making trust portable at a global scale. the core of ZeroTrust is enabling various autonomous entities to exchange their verified trust.
Authentications of individuals, suppliers, or smart objects are then no longer isolated, secluded, and secret. Zerotrust exposes successful authentication history to achieve the emergent effect of enhancing system resilience. Resilience is enhanced if one or more authentications are passport-grade level (e.g. eIDAS high). For instance, a ASML co-authored publication from 2014 about machine-to-machine communication states that "The MQTT protocol was not designed with security in mind". ZeroTrust is specifically designed to use passport-grade authentication as the foundation for M2M ecosystem security.

Experiments and performance analysis
Our contribution is a broad exploration of real-world systems in zero trust and performance analysis of our zero trust token. Our empirical contribution of this work consists of demystifying the security practices of huge companies. No complete zero trust solution exists which complete end-to-end openness, open source availability, and uncompromising self-hosting. We are therefore forced to explore the various required components. We combine these elements into "the ASML scenario". We use public available information from the largest public traded tech company in Europe to inform an extensive performance analysis of trust tokens.

First we quantify the most basic element: the wallet and passport-grade login service. Second we explore Time-based One-time Passwords (TOTP). Third, a hardware security module (HSM) is a physical computing device that safeguards and manages secrets (most importantly digital keys).
Finally we present an in-depth performance analysis of our trust token using ModeSecurity engine, based on quantitative data from "the ASML scenario".

@AdiDumi
Copy link

AdiDumi commented Nov 4, 2024

Thesis_new.pdf

  • thesis rewriting, changing direction and story for Introduction, Problem description and Design. keeping the information about zero trust that was already in the Introduction for now
  • Focus on cyberattacks and transparent global security
  • Still some questions regarding experiments mentioned above
  • Switched to full linux developement -> no more firewall problems in sending requests
  • Crashes encountered in running experiments at around 600 users -> after investigations found out that the system runs out of file descriptors for (1000) threads but I managed to temporarily increase their number in a terminal shell ✔️
  • New results with still no crashes even at 1000 running users -> but very slow responses
    test2
  • Writing left at polishing experiments/related work and adding conclusion/abstract

@synctext
Copy link
Member Author

synctext commented Nov 5, 2024

  • Wow, it seems to work quite well 😁 😁 😁
    • graduation in coming months 👏 🎉
  • Where is the president Biden executive order on Zero Trust?
  • Zero Trust Architecture (ZTA) only use that one term please. Replace earlier Zero Trust movement
  • Introduction is too long. Also explains the "Zero Trust architectural principles", separate section.
    • Introduction should be only the first page! Still include great explainer stuff like: This thesis presents TrustZero, a layer of zero-trust security designed to be applied at large scale. As a proofof-principle we created a universal ”trust token” that consists of a non-revocable self-sovereign identity with a list of trust attestations.
    • "Fig. 1: TrustZero architecture overview", needs a make-over.
    • New picture needed for Figure 1 on Page 2 ❕ (ZeroTrustGlobal core concept of trust tokens)
    • First page needs high-level overview: bad people on Internet, zero trust architecture, we expand that idea to the entire digital universe 😀, global trust, trust token, ModSecurity.
  • Related work: 1 related paper == 1 subsection is what bsc students tend to do. Abstract away to first principles or something fancy 🍻
  • More explicit design section naming like: "Our TrustZero Architecture"
  • Also mention in Introduction: this is between organisations (innovative!). First sentence of "design" section should also explain: it's for complex supply chains that. Then talk about Log4J, Crowdstrike.
    • Possible rename your core idea to ZeroTrustGlobal?
    • Could be seen as a "unified ledger for accountability"
    • Internet-of-Trust (IoT), "Zero Trust 2.0" 🤣
  • NEW section 4.1 "modern complex supply chain"
  • Courier fixed width font for header code SecRule &REQUEST HEADERS: User −Key− S i g n a t u r e s
  • "Fig. 2: First experiemnt" 2-column for readability

update: try out EBSI passport-grade wallet service about trust: https://github.com/eu-digital-identity-wallet/eudi-srv-web-trustprovider-signer-java ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

2 participants