diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..db12c9b --- /dev/null +++ b/.gitattributes @@ -0,0 +1 @@ +apt-cyg text eol=lf diff --git a/README.md b/README.md old mode 100644 new mode 100755 index 6e96d5a..b66318e --- a/README.md +++ b/README.md @@ -1,28 +1,211 @@ apt-cyg ======= -apt-cyg is a command-line installer for Cygwin which cooperates with Cygwin Setup and uses the same repository. The syntax is similar to apt-get. Usage examples: +apt-cyg is a command-line installer for [Cygwin](http://cygwin.com/) which cooperates with Cygwin Setup and uses the same repository. The syntax is similar to apt-get. -* "apt-cyg install " to install packages -* "apt-cyg remove " to remove packages -* "apt-cyg update" to update setup.ini -* "apt-cyg show" to show installed packages -* "apt-cyg find " to find packages matching patterns -* "apt-cyg describe " to describe packages matching patterns -* "apt-cyg packageof " to locate parent packages +Usage +----- + +### Command line + + apt-cyg [] [ [ ...]] + +### Subcommands + +| Subcommand | Description | +|:------- |:----------- | +| `install ` | to install packages | +| `remove ` | to remove packages | +| `update` | to update *setup.ini* | +| `show` | to show installed packages | +| `find ` | to find packages matching patterns | +| `describe ` | to describe packages matching patterns | +| `packageof ...` | to locate parent packages | +| `pathof {cache\|mirror\|mirrors\|mirrordir\|cache/mirrordir\|setup.ini}` | to show path | +| `key-add ...` | to add keys contained in \ | +| `key-del ...` | to remove keys \ | +| `key-list` | to list keys | +| `key-finger` | to list fingerprints | +| `upgrade-self` | to upgrade apt-cyg | +| `depends ...` | to show forward dependency information for packages with depth. | +| `rdepends ...` | to show reverse dependency information for packages with depth. | +| `completion-install` | to install completion. | +| `completion-uninstall` | to uninstall completion. | +| `mirrors-list` | to show list of mirrors. | +| `mirrors-list-long` | to show list of mirros with full details. | +| `mirrors-list-online` | to show list of mirrors from online. | +| `benchmark-mirrors ...` | to benchmark mirrors. | +| `benchmark-parallel-mirrors ...` | to benchmark mirrors in parallel. | +| `benchmark-parallel-mirrors-list` | to benchmark mirrors-list in parallel. | +| `scriptinfo` | to show script information. | +| `show-packages-busyness ...` | to show if packages are busy or not. | +| `dist-upgrade` | to upgrade all packages that is installed. This subcommand uses **`setup-*.exe`** | +| `update-setup` | to update setup.exe | +| `setup [ ...]` | to call setup.exe | +| `packages-total-count` | count number of total packages from setup.ini | +| `packages-total-size []` | count size of total packages from setup.ini | +| `packages-cached-count` | count number of cached packages in cache/mirrordir. | +| `packages-cached-size` | count size of cached packages in cache/mirrordir. | +| `repair-acl` | to repair the windows ACL (Access Control List). | +| `repair-postinstall` | Repair postinstall scripts. | +| `source ...` | download source archive. | +| `mirror-source ...` | download the source package into the current cache/mirrordir as mirror. | +| `download ...` | download the binary package into the current directory. | +| `mirror ...` | download the binary package into the current cache/mirrordir as mirror. | +| `browse-homepage-with-mirror-source [ ...]` | Browse homepages of packages with mirror-source. | +| `browse-homepage [ ...]` | Browse homepages of packages. | +| `browse-summary [ ...]` | Browse summaries of packages. | +| `listfiles ...` | List files 'owned' by package(s). | +| `get-proxy` | Get proxies for eval. | +| `ls-categories` | List categories. | +| `ls-pkg-with-category` | List packages with category. | +| `category ` | List all packages in given \.| +| `setuprc-get
` | Get section from **`setup.rc`**. | +| `set-cache []` | Set cache. | +| `set-mirror [ ...]` | Set mirrors. Note: `setup-x86{,_64}.exe` uses all of them but currently `apt-cyg` uses the first one only. | +| `mark-auto [ ...]` | Mark the given packages as automatically installed. | +| `mark-manual [ ...]` | Mark the given packages as manually installed. | +| `mark-showauto` |Print the list of automatically installed packages. | +| `mark-showmanual` | Print the list of manually installed packages. | +| `call [ [ ...]]` | Call internal function in apt-cyg. | +| `time [ [ ...]]` | Report time consumed to call internal function in apt-cyg. | +| `filelist []` | File list like apt-file list. | +| `filesearch []` | File search like apt-file search. | + +### Options + +| Option | Description | +|:------ |:----------- | +| `--ag` | use the silver searcher (currently work only at packageof subcommand) | +| `--benchmark-timeout ` | Truncate items that take longer than \ when benchmarking | +| `--ignore-case`, `-i` | ignore case distinctions for `` | +| `--force-remove` | force remove +| `--force-fetch-trustedkeys` | force fetch trustedkeys | +| `--force-update-packageof-cache` | force update packageof cache | +| `--no-verify`, `-X` | Don't verify setup.ini signatures | +| `--no-check-certificate` | Don't validate the server's certificate | +| `--no-update-setup` | Don't update setup.exe +| `--no-header` | Don't print header | +| `--proxy`, `-p {auto\|inherit\|none\|}` | set proxy (default: ${APT_CYG_PROXY:-auto}) | +| `--completion-get-subcommand` | get subcommand (for completion internal use) | +| `--completion-disable-autoupdate` | disable completion autoupdate | +| `--max-jobs`, `-j ` | Run \ jobs in parallel | +| `--mirror`, `-m ` | set mirror +| `--mirror-index`, `-M ` | choose mirror from last-mirror list. \ is 0-based index in last-mirror. | +| `--cache`, `-c ` | set cache | +| `--file`, `-f ` | read package names from \ | +| `--noupdate`, `-u` | don't update setup.ini from mirror | +| `--ipv4`, `-4` | wget prefer ipv4 | +| `--no-progress` | hide the progress bar in any verbosity mode | +| `--quiet`, `-q` | quiet (no output) | +| `--verbose`, `-v` | verbose | +| `--help` | Display help and exit | +| `--version` | Display version and exit | + +Requirements +------------ + +`apt-cyg` requires the Cygwin default environment and the additional *Cygwin* packages: + +`wget`, `ca-certificates`, `gnupg`, `libiconv` + +In **32bit** version of cygwin, `wget` requires an additional setting for the `ca-certificates` package. +Choose one of below settings. + + # 1. Create symbolic link for the default ca-directory of wget. + ln -s /usr/ssl /etc/ + + # or + # 2. Set ca-directory paramete in '/etc/wgetrc'. + echo "ca-directory = /usr/ssl/certs" >> /etc/wgetrc + + # or + # 3. Set ca-directory paramete in '~/.wgetrc'. + echo "ca-directory = /usr/ssl/certs" >> ~/.wgetrc + +Remarks: +Above additional settings for wget is not required for 64bit version of cygwin. +But, as of 2014-01-17, perhaps `ca-certificates` package makes fail of certification in 64bit version of cygwin with Windows 8. See below: + +* Known Problem / [2014-01-17: ca-certificates package is not setup correct at x86_64 with Windows 8.](#2014-01-17-ca-certificates-package-is-not-setup-correct-at-x86_64-with-windows-8) Quick start ----------- +The most recommended way to deploy this fork can be seen from a link below: +* New features / [Upgrade apt-cyg](#upgrade-apt-cyg) + apt-cyg is a simple script. Once you have a copy, make it executable: - # chmod +x /bin/apt-cyg + chmod +x /bin/apt-cyg Optionally place apt-cyg in a bin/ folder on your path. Then use apt-cyg, for example: - # apt-cyg install nano + apt-cyg install nano + +New features +------------ + +### dist-upgrade support + +This fork has achieved `dist-upgrade` command by using `setup-x86.exe` and `setup-x86_64.exe` as a backend. +Note that all of running tasks on the cygwin will be killed before starting dist-upgrade. + +### Multiple hash algorithms support + +After the middle of 2015-03, the cygwin project changed the hash algorithm for checking tarball from md5 to sha512. +But, as of 2015-04-09, the cygwinports project seems still using md5. +This fork is available for both of cygwin and cygwinports by supporting algorithm of md5, sha1, sha224, sha256 and sha512. + +### Signature check and key management by GnuPG + +The default action of apt-cyg has been changed to check signature for 'setup.ini'. +Of course you can also avoid signature check by using `--no-verify` or `-X` options. +Public keys of cygwin and cygwinports are already registered to trusted keys of embeded. +If you want to use some other public keys, please use `key-*` subcommands. + +### Upgrade apt-cyg + +If apt-cyg is under Git version control, this fork can upgrade itself by `upgrade-self` subcommand. +Therefore, the most recommended way to deploy this fork is `copy and paste` below commands to cygwin console: + + # cd $DIR # Change working directory where you want to install apt-cyg + git clone https://github.com/kou1okada/apt-cyg.git + ln -s "$(realpath apt-cyg/apt-cyg)" /usr/local/bin/ + +If you want to use another fork, which forked from https://github.com/kou1okada/apt-cyg, rewrite the URL for apropriate one. + +### Proxy support + +Use `--proxy`, `-p` option. +This option must take a parameter from one of "auto", "inherit", "none" and URL. + +* "auto" will determine a proxy using a part of the [Web Proxy Auto-Discovery Protocol (WPAD)](http://en.wikipedia.org/wiki/Web_Proxy_Autodiscovery_Protocol). +The current implementation will look for a string of "PROXY URL" from "http://wpad/wpad.dat". +If "wpad.dat" could not be downloaded, the proxy settings are inherited from the parent environment. +* "inherit" will inherit the proxy settings from the parent environment. +* "none" will not use the proxy. +* URL can take a string like "protocol://hostname:port". + +For example: + + apt-cyg --proxy http://proxy.home:8080 update + +The default parameter is "${APT\_CYG\_PROXY:-auto}". +At the environment where is not provided the WPAD server, it makes the lag for a few seconds at startup. +So, if you don't want to use WPAD, please define APT\_CYG\_PROXY environment variable as below: + + export APT_CYG_PROXY=inherit + +### Bash completion support + +Bash completion script can be installed to "/etc/bash_completion.d/apt-cyg" by `completion-install` subcommand. +It will be automatically updated when apt-cyg is upgraded to newer version. +If you don't want to update it automatically, execute `completion-install` subcommand in conjunction with `--completion-disable-autoupdate` option. And `completion-uninstall` subcommand removes "/etc/bash_completion.d/apt-cyg". + +Some other forks, [Milly / apt-cyg](https://github.com/Milly/apt-cyg) under the cfg / apt-cyg fork, [ashumkin / apt-cyg](https://github.com/ashumkin/apt-cyg) and etc, are also supported it. Contributing ------------ @@ -30,3 +213,29 @@ Contributing This project has been re-published on GitHub to make contributing easier. Feel free to fork and modify this script. The [Google Code project](https://code.google.com/p/apt-cyg/) has a list of open issues. + +### Forks on the github + +Caution: +Please do not merge forks that have incompatible licenses. + +Ex.) Merging to the GPL from the MIT is possible. But merging to the MIT from the GPL is impossible. + +See [other_forks.md](other_forks.md) + +Todo +------------ + +- [ ] Support multi mirrors: Cygwin setup can use multi mirrors. They are recorded at last-mirror section in '/etc/setup/setup.rc'. It's useful for using [Cygwinports](http://cygwinports.org/). +- [ ] Support upgrade: But maybe, busy resources can not be upgraded, and rebase problem will happen. Cygwin setup resolves by replacing them at next reboot. +- [ ] Support dependency check for remove subcommand. + +Known Problems +------------ + +For older known problems see [known_problems.md](known_problems.md). + +License +------- + +[![The MIT license](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE) diff --git a/apt-cyg b/apt-cyg old mode 100644 new mode 100755 index 59e9522..2f92ede --- a/apt-cyg +++ b/apt-cyg @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # apt-cyg: install tool for cygwin similar to debian apt-get @@ -25,455 +25,2640 @@ # THE SOFTWARE. # +shopt -s extglob + +# EMBED_BEGIN: hhs_embed + +(( 5 <= DEBUG )) && set -x + +SCRIPT_PATH="$0" +SCRIPT_FILE="${SCRIPT_PATH##*/}" +SCRIPT_NAME="${SCRIPT_FILE%.*}" +SCRIPT_DIR="${SCRIPT_PATH%/*}" +SCRIPT_REALPATH="$(readlink -f "$SCRIPT_PATH")" +SCRIPT_REALFILE="${SCRIPT_REALPATH##*/}" +SCRIPT_REALNAME="${SCRIPT_REALFILE%.*}" +SCRIPT_REALDIR="${SCRIPT_REALPATH%/*}" + +# EMBED_BEGIN: hhs_sgr.bash + +function init_SGR () +{ + if [ -n "$COLORIZE" ]; then + SGR_reset="\e[0m" + SGR_bold="\e[1m" + SGR_fg_red="\e[31m" + SGR_fg_green="\e[32m" + SGR_fg_yellow="\e[33m" + SGR_fg_blue="\e[34m" + SGR_fg_magenta="\e[35m" + else + unset SGR_reset SGR_bold SGR_red SGR_green SGR_yellow SGR_blue SGR_magenta + fi + FATAL_COLOR="${SGR_fg_magenta}${SGR_bold}" + ERROR_COLOR="${SGR_fg_red}${SGR_bold}" + WARNING_COLOR="${SGR_fg_yellow}${SGR_bold}" + INFO_COLOR="${SGR_fg_green}${SGR_bold}" + DEBUG_COLOR="${SGR_fg_blue}${SGR_bold}" +} + +# EMBED_END: hhs_sgr.bash +# EMBED_BEGIN: hhs_debug.bash + +THRESHOLD_OF_FATAL=0 +THRESHOLD_OF_ERROR=1 +THRESHOLD_OF_WARNING=2 +THRESHOLD_OF_INFO=3 +THRESHOLD_OF_DEBUG=4 + +function abort () #= [EXITCODE=1 [CALLSTACKSKIP=0]] +{ + dump_callstack $(( 2 + ${2:-0} )) + exit "${1:-1}" +} + +function dump_callstack () #= [N=1] +#? N is a depth to skip the callstack. +#? $CALLSTACK_SKIP is depth to skip the callstack too. +#? N and CALLSTACK_SKIP is summed before skipping. +#? The default value is N = 1 and CALLSTACK_SKIP = 0. +{ + local i + echo "Callstack:" + for i in `seq "$(( ${#FUNCNAME[@]} - 1 ))" -1 $(( ${1:-1} + ${CALLSTACK_SKIP:-0} ))`; do + echo -e "\t${BASH_SOURCE[i]}: ${FUNCNAME[i]}: ${BASH_LINENO[i-1]}" + done +} #/dump_callstack + +function source_at () #= [N=1] +{ + local i=${1:-1} + echo -e "${DEBUG_COLOR}at :${SGR_reset} ${BASH_SOURCE[i-1]}: ${FUNCNAME[i]}: ${BASH_LINENO[i-1]}" +} + +function fatal () #= [MESSAGES ...] +{ + (( THRESHOLD_OF_FATAL <= ${VERBOSE:-0} )) || return 1 + echo -e "${FATAL_COLOR}Fatal:${SGR_reset} $@" + (( THRESHOLD_OF_FATAL <= SOURCE_AT )) && source_at 2 +} >&2 #/fatal + +function error () #= [MESSAGES ...] +{ + (( THRESHOLD_OF_ERROR <= ${VERBOSE:-0} )) || return 1 + echo -e "${ERROR_COLOR}Error:${SGR_reset} $@" + (( THRESHOLD_OF_ERROR <= SOURCE_AT )) && source_at 2 +} >&2 #/error + +function warning () #= [MESSAGES ...] +{ + (( THRESHOLD_OF_WARNING <= ${VERBOSE:-0} )) || return 1 + echo -e "${WARNING_COLOR}Warning:${SGR_reset} $@" + (( THRESHOLD_OF_WARNING <= SOURCE_AT )) && source_at 2 +} >&2 #/warning + +function info () #= [MESSAGES ...] +{ + (( THRESHOLD_OF_INFO <= ${VERBOSE:-0} )) || return 1 + echo -e "${INFO_COLOR}Info:${SGR_reset} $@" + (( THRESHOLD_OF_INFO <= SOURCE_AT )) && source_at 2 +} >&2 #/info + +function debug () #= [MESSAGES ...] +{ + (( THRESHOLD_OF_DEBUG <= ${VERBOSE:-0} )) || return 1 + echo -e "${DEBUG_COLOR}Debug:${SGR_reset} $@" + (( THRESHOLD_OF_DEBUG <= SOURCE_AT )) && source_at 2 +} >&2 #/debug + +# EMBED_END: hhs_debug.bash + +: ${VERBOSE:=$THRESHOLD_OF_WARNING} +: ${SOURCE_AT:=$THRESHOLD_OF_WARNING} +: ${COLORIZE:=1} +init_SGR + +# EMBED_END: hhs_embed + +TRUSTEDKEYS=( CYGWIN ); +# ./pubring.asc +# ------------ +# pub 4096R/E2E56300 2020-02-27 [expires: 2026-02-27] +# uid Cygwin +TRUSTEDKEY_CYGWIN_SUM="3666810719f036011944cd6e95f6e73a64be9fffca06c674f2623172e044d5a5eb53fa28d4922429a8c84bd0b8e9fc2d7c4ee0e300d11b9cf934b3c68c76811d" +TRUSTEDKEY_CYGWIN_FPR="56405CF6FCC81574682A5D561A698DE9E2E56300" +TRUSTEDKEY_CYGWIN_URL_LATEST="https://cygwin.com/key/pubring.asc" + # this script requires some packages -WGET=`which wget 2> /dev/null` -BZIP2=`which bzip2 2> /dev/null` -TAR=`which tar 2> /dev/null` -GAWK=`which awk 2> /dev/null` -if test "-$WGET-" = "--" || test "-$BZIP2-" = "--" || test "-$TAR-" = "--" \ - || test "-$GAWK-" = "--" -then - echo You must install wget, tar, gawk and bzip2 to use apt-cyg. +read WGET < <( type -p wget 2>/dev/null ) +read TAR < <( type -p tar 2>/dev/null ) +read GAWK < <( type -p awk 2>/dev/null ) +read ICONV < <( type -p iconv 2>/dev/null ) +read GPG < <( type -p gpg2 2>/dev/null || type -p gpg 2>/dev/null ) +if [ -z "$WGET" -o -z "$TAR" -o -z "$GAWK" -o -z "$ICONV" ]; then + echo You must install wget, tar, gawk and libiconv to use apt-cyg. exit 1 fi +function usage() +{ + cat<<-EOD + Usage: apt-cyg [] [ [ ...]] + Installs and removes Cygwin packages. + Subcommands: + install : to install packages + remove : to remove packages + update : to update setup.ini + show : to show installed packages + find ... : to find packages matching patterns + describe ... : to describe packages matching patterns + packageof ... : + to locate parent packages + pathof {cache|mirror|mirros|mirrordir|cache/mirrordir|setup.ini} : + to show path + key-add ... : to add keys contained in + key-del ... : to remove keys + key-list : to list keys + key-finger : to list fingerprints + upgrade-self : to upgrade apt-cyg + depends ... : + to show forward dependency information + for packages with depth. + rdepends ... : + to show reverse dependency information + for packages with depth. + completion-install : to install completion. + completion-uninstall : to uninstall completion. + mirrors-list : to show list of mirros. + mirrors-list-long : to show list of mirros with full details. + mirrors-list-online : to show list of mirros from online. + benchmark-mirrors ... : + to benchmark mirrors. + benchmark-parallel-mirrors ... : + to benchmark mirrors in parallel. + benchmark-parallel-mirrors-list : + to benchmark mirrors-list in parallel. + scriptinfo : to show script information. + show-packages-busyness ... : + to show packages are busy or not. + dist-upgrade : to upgrade all packages that is installed. + This subcommand uses setup.exe + update-setup : to update setup.exe + setup [ ...] : to call setup.exe + packages-total-count : count number of total packages from setup.ini + packages-total-size [] : + count size of total packages from setup.ini + packages-cached-count : count number of cached packages + in cache/mirrordir. + packages-cached-size : count size of cached packages + in cache/mirrordir. + repair-acl : repair acl. + repair-postinstall : Repair postinstall scripts. + source ... : + download source archive. + mirror-source ... : + download the source package + into the current cache/mirrordir as mirror. + download ... : + download the binary package + into the current directory. + mirror ... : + download the binary package + into the current cache/mirrordir as mirror. + browse-homepage-with-mirror-source [ ...] : + Browse homepages of packages with mirror-source. + browse-homepage [ ...] : + Browse homepages of packages. + browse-summary [ ...] : + Browse summaries of packages. + listfiles [ ...] : + List files 'owned' by package(s). + get-proxy : Get proxies for eval. + ls-categories : List categories. + ls-pkg-with-category : List packages with category. + category : List all packages in given . + setuprc-get
: Get section from 'setup.rc'. + set-cache [] : Set cache. + set-mirror [ ...] : + Set mirror. + Note: setup-x86{,_64}.exe uses all of them + but currently apt-cyg uses the first one only. + mark-auto [ ...] : + Mark the given packages + as automatically installed. + mark-manual [ ...] : + Mark the given packages as manually installed. + mark-showauto : Print the list of + automatically installed packages. + mark-showmanual : Print the list of manually installed packages. + call [ [ ...]] : + Call internal function in apt-cyg. + time [ [ ...]] : + Report time consumed + to call internal function in apt-cyg. + filelist [] : File list like apt-file list. + filesearch [] : File search like apt-file search. + Options: + --ag : use the silver searcher + (currently work only at packageof subcommand) + --benchmark-timeout : + Truncate items that take longer than + when benchmarking. + --ignore-case, -i : ignore case distinctions for + --force-remove : force remove + --force-fetch-trustedkeys : + force fetch trustedkeys + --force-update-packageof-cache : + force update packageof cache + --no-verify, -X : Don't verify setup.ini signatures + --no-check-certificate : Don't validate the server's certificate + --no-update-setup : Don't update setup.exe + --no-header : Don't print header + --proxy, -p {auto|inherit|none|} : + set proxy (default: \${APT_CYG_PROXY:-auto}) + --completion-get-subcommand : + get subcommand (for completion internal use) + --completion-disable-autoupdate : + disable completion autoupdate + --max-jobs, -j : Run jobs in parallel + --mirror, -m : set mirror + --mirror-index, -M : choose mirror from last-mirror list. + is 0-based index in last-mirror. + --cache, -c : set cache + --file, -f : read package names from + --noupdate, -u : don't update setup.ini from mirror + --ipv4, -4 : wget prefer ipv4 + --no-progress : hide the progress bar in any verbosity mode + --quiet, -q : quiet (no output) + --verbose, -v : verbose + --help : Display help and exit + --version : Display version and exit + EOD +} + + + +function git_status () +{ + local origin status + [ ! -d "${SCRIPT_REALDIR%/}/.git" ] && return 1 + pushd "${SCRIPT_REALDIR}" >/dev/null + echo + origin="$(git config --get remote.origin.url)" && echo "origin: $origin" + echo "commit: $(git log -1 --pretty=format:"%ai %h%d")" + status="$(git status --short "${SCRIPT_REALNAME}")" + [ -n "$status" ] && echo "status: $status" + popd >/dev/null +} + +function version () +{ + cat<<-EOD + kou1okada/apt-cyg + forked from transcode-open/apt-cyg$(git_status) + + This script is based on apt-cyg version 0.57 + Copyright (c) 2005-9 Stephen Jungels. Released under the GPL. + Copyright (c) 2013-7 Stephen Jungels. Republished under the MIT license. + EOD +} + +# Usage: verbose [level [msg ...]] +function verbose () +{ + (( OPT_VERBOSE_LEVEL < "${1:-1}" )) && return + (( 1 < $# )) && echo "${@:2}" || cat +} >&2 + +function update_verbosefor () +# Assign /dev/fd/{1000..1005} for ${verbosefor[0..5]} +{ + local i + for i in {0..5}; do + (( OPT_VERBOSE_LEVEL < i )) && eval exec "100$i>/dev/null" || eval exec "100$i>&2" + done +} + +function detect_field_width () +# Read stdin and detect field width. +# Return NF+2 values as follorwing: +# w(0) w(1) w(2) ... w(NF) sum(w(1),w(2), ..., w(NF))+NF-1 +# Now NF is a number of fields, +# w(x) is a function of maxium width of field x th +# (note that 0 does not mean the field but the whole line) +# and sum(v1,v2, ..., vn) is a function to sum all arguments. +{ + awk ' + function max(a,b){return a [ ...] +# Join strings with separator. +# Note that separator must be single character. +# If you want separator with multiple character, use join_str_ex. +{ + (IFS="$1"; printf "%s" "${*:2}") +} + +function join_str_ex () # [ ...] +# Join strings with separator. +# Different for join_str the separator can take multiple character. +{ + local str="$(join_str $'\x1c' "${@:2}")" + printf "%s" "${str//$'\x1c'/$1}" +} + +function split_str () # [ ...] +# Split strings with separator. +{ + local str=( "${@:2}" ) + (IFS=$'\n'; printf "%s" "${str[*]//$1/$'\n'}") +} + +function join_str_uniq () # [ ...] +# Append values to head of joined strings. +{ + local str + readarray -t str < <(split_str "$@" | uniqex) + join_str "$1" "${str[@]}" +} + +function mkdirp () # +{ + [ -d "$1" ] || mkdir -p "$1" || { error "mkdir failed: $1"; exit 1; } +} + +function mktmpfn () # -v +# Temporary filename +{ + printf ${@:1:2} "/tmp/${SCRIPT_NAME}.$$.%04x%04x" $RANDOM $RANDOM +} + +function init_comspec () +{ + : ${SYSTEMPATH:=$(join_str_uniq : "$(cygpath -u "${SYSTEMROOT:-$WINDIR}")"{/system32,} "$PATH")} +} + +function comspec () # [ ...] +# Call $COMSPEC. +{ + init_comspec + PATH="$SYSTEMPATH" "$(cygpath "$COMSPEC")" "$@" +} + +function cygstart () # [ ...] +{ + init_comspec + PATH="$SYSTEMPATH" cygstart.exe "$@" +} + +function update_cache_for_mirrors_list_online () +{ + pushd "$apt_cyg_cachedir" >/dev/null + wget -qN https://cygwin.com/mirrors.lst + popd >/dev/null +} + +function is_official_mirrors_of_cygwin () # ... +# Check whether are listed in official mirrors list of cygwin. +# Official mirros list provides on https://cygwin.com/mirrors.html. +# Args: +# : URLs of the cygwin mirror. +# Return: +# Return zero if all mirrors are known, non-zero otherwise. +{ + local mirror + local result=0 + local local_list="$(apt-cyg-mirrors-list)" + local online_list="$(apt-cyg-mirrors-list-online)" + for mirror; do + if ! grep -q "$mirror" <<< "$local_list"; then + warning "/etc/setup/setup.rc doesn't know your mirror: ${SGR_bold}$mirror${SGR_reset}" >&2 + result=1 + fi + if ! grep -q "$mirror" <<< "$online_list"; then + warning "Official mirrors.lst doesn't know your mirror: ${SGR_bold}$mirror${SGR_reset}" >&2 + result=1 + fi + done + return $result +} + +function mirror_to_mirrordir () # +{ + local tmp="${1//:/%3a}" + echo "${tmp//\//%2f}" +} + +function get_utf8_setuprc () +{ + local utf8_setuprc="$apt_cyg_cachedir/setup.rc.utf8" + mkdirp "$apt_cyg_cachedir" + [ "$utf8_setuprc" -nt /etc/setup/setup.rc ] && { + cat "$utf8_setuprc" + } || { + cp2utf8 /etc/setup/setup.rc | tee "$utf8_setuprc" + } +} + +function setuprc_import_sections () #
... +{ + local IFS="|" + get_utf8_setuprc \ + | awk -vRS='\n\\<|\n\\'\' -vFS='\n\t' -vsections="^${*//\\/\\\\}$" ' + match($1, sections) { + s = gensub(/-/, "_", "g", $1) "=(" + for(i = 2; i <= NF; i++) s = s " \x27" $i "\x27"; + s = s " )"; + print s; + } + ' +} + +function setuprc_get_section () #
+{ + get_utf8_setuprc \ + | awk -vRS='\n\\<|\n\\'\' -vFS='\n\t' -vsection="${1//\\/\\\\}" ' + $1 == section {for(i = 2; i <= NF; i++) print $i;} + ' +} + +function setuprc_set_section () #
[ ...] +{ + local s="$(join_str_ex $'\n\t' "${@}")" + local setuprc="/etc/setup/setup.rc" + local setuprc_bak="${setuprc},$(date -r "$setuprc" "+%Y%m%d_%H%M%S")" + + cp -a "$setuprc" "$setuprc_bak" + + cp2utf8 "$setuprc_bak" \ + | awk -vRS='\n\\<|\n\\'\' -vFS='\n\t' -vsection="${1//\\/\\\\}" -vs="${s//\\/\\\\}" ' + $1 != section + $1 == section {print s; done=1;} + END {if(!done) print s;} + ' \ + | utf82cp > "$setuprc" +} + +function set_cachefile () # +# Set filename of cachevars to $cachefile. +# Returns: +# $cachefile : Overwrite with filename of cachevars. +# Note: +# This is internal function for load_vars_from_cache and save_vars_to_cache. +{ + cachefile="$apt_cyg_cachedir/cache_for_vars_of_${FUNCNAME[2]}" +} + +function load_vars_from_cache () # [ ...] || { init_vars ...; save_vars_to_cache [ ...]; } +# restore vars from cache. +{ + local cachefile; set_cachefile + local i + set -- "$SCRIPT_PATH" "$@" + for i; do [ "$cachefile" -nt "$i" ] || return 1; done + source "$cachefile" +} + +function save_vars_to_cache () # [ ...] +# save vars to cache. +{ + local cachefile; set_cachefile + declare -p "$@" | sed -E 's/^declare .. //g' >"$cachefile" +} + +function findworkspace() +{ + local cachevars=( last_cache last_mirror mirror cache arch mirrordir ) + load_vars_from_cache /etc/setup/setup.rc || { + eval "$(setuprc_import_sections last-cache last-mirror)" + + mirror=( "${last_mirror[@]}" ) + cache="$(cygpath -au "$last_cache")" + + cache=( "${cache[@]/%*(\/)/}" ) + mirror=( "${mirror[@]/%*(\/)/}" ) + + readarray -t mirrordir < <(mirror_to_mirrordir "${mirror[@]/%/\/}" ) + + save_vars_to_cache "${cachevars[@]}" + } + [ -z "$OPT_MIRROR_INDEX" ] || { + [[ $OPT_MIRROR_INDEX =~ ^[0-9]+$ ]] || { error "--mirror-index, -M must be 0 <= n: n = $OPT_MIRROR_INDEX"; exit 1; } + mirror=( "${mirror[@]:OPT_MIRROR_INDEX:1}" "${mirror[@]:0:OPT_MIRROR_INDEX}" "${mirror[@]:OPT_MIRROR_INDEX+1}" ) + readarray -t mirrordir < <(mirror_to_mirrordir "${mirror[@]/%/\/}" ) + } + + verbose 1 "Cache directory is $cache" + verbose 1 "Mirror is $mirror" + + mkdirp "$cache/$mirrordir/$arch" + cd "$cache/$mirrordir/$arch" + + init_gnupg + fetch_trustedkeys +} + +function download_and_verify () # +{ + local urls=( "$1"{,.sig} ) + wget -N "${urls[@]}" || return 1 + if [ -z "$no_verify" ]; then + [ -e "${1##*/}.sig" ] && verify_signatures "${1##*/}.sig" || return 1 + fi + [ -e "${1##*/}" ] +} + +function files_backup () +{ + local file + for file; do + [ -e "${file}~" ] && mv "${file}~" "${file}" + [ -e "${file}" ] && cp -a "${file}" "${file}~" + done +} + +function files_restore () +{ + local file + for file; do + [ -e "${file}" ] && rm "${file}" + [ -e "${file}~" ] && mv "${file}~" "${file}" + done +} + +function files_backup_clean () +{ + local file + for file; do + [ -e "${file}~" ] && rm "${file}~" + done +} + +function setupini_download () +{ + local BASEDIR="$cache/$mirrordir/$arch" + mkdirp "$BASEDIR" + + [ $noscripts -ne 0 -o $noupdate -ne 0 ] && return + + pushd "$BASEDIR" > /dev/null + files_backup setup.{zst,xz,bz2,ini}{,.sig} + + while true; do + verbose 1 "Updating setup.ini" + false \ + || { download_and_verify "$mirror/$arch/setup.zst" && { zstd -dfkq setup.zst && mv setup{,.ini} && rm -f setup.ini.sig || ! rm -fv setup.zst; }; } \ + || { download_and_verify "$mirror/$arch/setup.xz" && { xz -dfk setup.xz && mv setup{,.ini} && rm -f setup.ini.sig || ! rm -fv setup.xz; }; } \ + || { download_and_verify "$mirror/$arch/setup.bz2" && { bzip2 -dfk setup.bz2 && mv setup{,.ini} && rm -f setup.ini.sig || ! rm -fv setup.bz2; }; } \ + || download_and_verify "$mirror/$arch/setup.ini" || break + + files_backup_clean setup.{zst,xz,bz2,ini}{,.sig} + popd > /dev/null + verbose 1 "Updated setup.ini" + return + done + files_restore setup.{zst,xz,bz2,ini}{,.sig} + popd > /dev/null + error "updating setup.ini failed, reverting." + return 1 +} + +function getsetup () +{ + setupini_download || return 1 +} + +function checkpackages() +{ + if [ $# -eq 0 ]; then + echo Nothing to do, exiting + exit 0 + fi +} + +function init_gnupg () +{ + [ -z "$GPG" -o -n "$no_verify" ] && return + export GNUPGHOME="$cache/.apt-cyg" + if [ ! -d "$GNUPGHOME" ]; then + if ! { mkdir -p "$GNUPGHOME" && chmod 700 "$GNUPGHOME"; } then + error "Cannot initialize directory: $GNUPGHOME" + exit 1 + fi + fi +} + +# Usage: ask_user [MESSAGE [OPTIONS]] +function ask_user () +{ + local answer retcode option + local MESSAGE="$1" + local OPTIONS="${2:-y/N}" + local DEFAULT="$(echo "$OPTIONS" | awk -v FS=/ '{for(i=1;i<=NF;i++)if(match(substr($i,1,1),/[A-Z]/)){print $i; exit}}')" + local SPLIT_OPTIONS + readarray -t SPLIT_OPTIONS < <(echo "$OPTIONS" | sed -e 's:/:\n:g') + while true; do + echo -n "${MESSAGE}${MESSAGE:+ }[${OPTIONS}] " + read answer + retcode=0 + for option in "${SPLIT_OPTIONS[@]}"; do + if [ "$option" = "${answer:-$DEFAULT}" ]; then + return $retcode + fi + retcode=$(( retcode + 1 )) + done + done +} + +unset INIT_WGET +function init_wget () +{ + [ -n "$INIT_WGET" ] && return + + eval "$( + "$WGET" --help \ + |& awk ' + /--show-progres/{print "HAVE_SHOW_PROGRESS=--show-progress"} + /--no-verbose/ {print "HAVE_NO_VERBOSE=--no-verbose"} + ')" + (( OPT_VERBOSE_LEVEL < 0 )) && WGET+=( --quiet ) || \ + (( OPT_VERBOSE_LEVEL < 1 )) && WGET+=( --quiet $HAVE_SHOW_PROGRESS ) || \ + (( OPT_VERBOSE_LEVEL < 2 )) && WGET+=( $HAVE_NO_VERBOSE $HAVE_SHOW_PROGRESS ) + + proxy_setup + + INIT_WGET=DONE +} + +function wget () +{ + init_wget + "${WGET[@]}" "$@" +} + +# Reference: +# https://www.gnu.org/software/wget/manual/wget.html#Exit-Status +# Usage: wget-exitstatus EXITSTATUS +function wget-exitstatus () +{ + case $1 in + 0) echo "No problems occurred.";; + 1) echo "Generic error code.";; + 2) echo "Parse error?for instance, when parsing command-line options, the '.wgetrc' or '.netrc'...";; + 3) echo "File I/O error.";; + 4) echo "Network failure.";; + 5) echo "SSL verification failure.";; + 6) echo "Username/password authentication failure.";; + 7) echo "Protocol errors.";; + 8) echo "Server issued an error response.";; + *) echo "Unknown errors.";; + esac +} + +function wget_advice () +{ + local status=${1:-$?} + case $status in + 5) echo "If you can tolerate security risks, use --no-certification option." ;; + esac + return $status +} + +# Usage: get_advice cmd +function get_advice () +{ + local status=${2:-$?} + type "${1}_advice" >&/dev/nul && { ( exit $status ); "${1}_advice"; } + return $status +} + +# Usage: push_advice cmd +function push_advice () +{ + local status=${2:-$?} + ADVICE+=( "$( get_advice "${1}_advice")" ) + return $status +} + +function show_advice () +{ + local i + for i in "${ADVICE[@]}"; do + echo "$i" >&2 + done +} + +# Usage: wget_and_hash_check label hash url file +function wget_and_hash_check () +{ + local LABEL="$1" + local SUM="$2" + local URL="$3" + local FILE="$4" + if ! { wget "$URL" -O "$FILE" || get_advice wget; }; then + echo "$LABEL: FAILED: Could not download $URL." + return 1 + fi + if ! hash_check <<<"$SUM *$FILE" >&/dev/null; then + echo "$LABEL: FAILED: Hash does not match $URL." + return 2 + fi + echo "$LABEL: OK" +} + +function get_gpg_fingerprint_to_assoc () # +{ + local -n result="$1" + local line lines + readarray -t lines < <("${GPG[@]}" --with-colons --fingerprint) + for line in "${lines[@]}"; do + [[ "$line" =~ ^fpr:+([0-9A-Za-z]+):* ]] && result[${BASH_REMATCH[1]}]=1 + done +} + +function fetch_trustedkeys () +{ + [ -z "$GPG" -o -n "$no_verify" ] && return + local i + local FILE ; mktmpfn -v FILE + local FILE_LATEST; mktmpfn -v FILE_LATEST + local -A FPRS_assoc; get_gpg_fingerprint_to_assoc FPRS_assoc + for i in "${TRUSTEDKEYS[@]}"; do + local LABEL="TRUSTEDKEY_${i}" + local -n SUM="${LABEL}_SUM" + local -n FPR="${LABEL}_FPR" + local -n URL="${LABEL}_URL" + local -n URL_LATEST="${LABEL}_URL_LATEST" + local CASE="" + if [ -z "$force_fetch_trustedkeys" -a -n "${FPRS_assoc[$FPR]}" ]; then + continue + fi + if [ -n "$URL" ]; then + wget_and_hash_check "$LABEL" "$SUM" "$URL" "$FILE" + CASE+="$?" + else + CASE+="-" + fi + if [ -n "$URL_LATEST" ]; then + wget_and_hash_check "$LABEL" "$SUM" "$URL_LATEST" "$FILE_LATEST" + CASE+="$?" + else + CASE+="-" + fi + case "$CASE" in + 00|01|0-) + "${GPG[@]}" --import "$FILE" + ;; + 02) + warning "${LABEL} has been updated." + "${GPG[@]}" --import "$FILE" + ;; + -0) + "${GPG[@]}" --import "$FILE_LATEST" + ;; + 10|20) + error "${LABEL} has miss configuration." + exit 1 + ;; + 11|1-|-1) + error "Could not download ${LABEL}." + exit 1 + ;; + 12|-2) + error "${LABEL} has been updated, maybe. But sometimes it may has been cracked. Be careful !!!" + exit 1 + ;; + 21|22|2-) + error "${LABEL} has been cracked, maybe" + exit 1 + ;; + --) + error "${LABEL} has no URL." + exit 1 + ;; + esac + rm "$FILE" "$FILE_LATEST" &>/dev/null + done +} + +# Usage: verify_signatures files ... +function verify_signatures () +{ + while [ $# -gt 0 ]; do + if ! "${GPG[@]}" --verify "$1" 1>&1002 2>&1002; then + error "BAD signature: $1" + return 1 + else + verbose 0 -e "${SGR_fg_green}${SGR_bold}signature verified:${SGR_reset} $1" + fi + shift + done +} + +# Usage: apt-cyg-key-add pkey ... +function apt-cyg-key-add () +{ + [ -z "$GPG" ] && { error "GnuPG is not installed. Prease install gnupg package"; exit 1; } + local pkeys + for pkey; do + pkeys+=( "$(cygpath -a "$pkey" )" ) + done + findworkspace + for pkey in "${pkeys[@]}"; do + "${GPG[@]}" --import "$pkey" + done +} + +# Usage: apt-cyg-key-add keyid ... +function apt-cyg-key-del () +{ + [ -z "$GPG" ] && { error "GnuPG is not installed. Prease install gnupg package"; exit 1; } + local keyid + findworkspace + for keyid; do + "${GPG[@]}" --batch --yes --delete-key "$keyid" + done +} + +function apt-cyg-key-list () +{ + [ -z "$GPG" ] && { error "GnuPG is not installed. Prease install gnupg package"; exit 1; } + findworkspace + "${GPG[@]}" --list-keys +} + +function apt-cyg-key-finger () +{ + [ -z "$GPG" ] && { error "GnuPG is not installed. Prease install gnupg package"; exit 1; } + findworkspace + "${GPG[@]}" --fingerprint +} + +function apt-cyg-pathof () +{ + OPT_VERBOSE_LEVEL=0 findworkspace + while [ "$#" -gt 0 ]; do + case "$1" in + cache) echo "$cache" ;; + mirror) echo "$mirror" ;; + mirrors) printf "%s/\n" "${mirror[@]}" ;; + mirrordir) echo "$mirrordir" ;; + cache/mirrordir) echo "$cache/$mirrordir" ;; + setup.ini) echo "$cache/$mirrordir/$arch/setup.ini" ;; + *) + error "unknown parameter: $1" + exit 1 + ;; + esac + shift + done +} + +function upgrade-self-with-git () +{ + if [ ! -d "$SCRIPT_REALDIR/.git" ]; then + warning "apt-cyg is not under the git version control." + return 1 + fi + proxy_setup + pushd "$SCRIPT_REALDIR" > /dev/null + git pull -v + popd > /dev/null +} + +function upgrade-self-with-wget () +{ + local updated_url='https://raw.githubusercontent.com/kou1okada/apt-cyg/master/apt-cyg' # TODO: Do not use MAGIC NUMBER + local temp_file; mktmpfn -v temp_file + wget "${updated_url}" -q -O "$temp_file" + chmod +x "$temp_file" + { diff "$temp_file" "$SCRIPT_REALPATH" >/dev/null && exit 0; } || cp "$temp_file" "$SCRIPT_REALPATH" + rm "$temp_file" +} + +function apt-cyg-upgrade-self () +{ + upgrade-self-with-git && return + echo "Fall back to wget for upgrade-self." + upgrade-self-with-wget +} + +function proxy_auto () +{ + local hash=( $(ipconfig |& md5sum -b) ) + local cache="/tmp/apt-cyg.proxy.$hash" + local last="$(stat -c %Y "$cache" 2>/dev/null)" + local now="$(printf "%(%s)T")" + local proxy + + [ -n "$OPT_PROXY_FORCE_REFRESH" ] && last=0 + if (( (now - ${last:-0}) < OPT_PROXY_REFRESH_INTERVAL )); then + read proxy < "$cache" + else + read proxy < <( + "${WGET[@]}" --no-proxy -q -O - wpad/wpad.dat \ + | sed -nE 's@//.*@@g;/PROXY/s@^.*PROXY\s*([^"; ]*).*$@http://\1@gp' + ) + echo "$proxy" > "$cache" + fi + [ -n "$proxy" ] && proxy_set "$proxy" +} + +function proxy_set () +{ + export http_proxy="$1" + export https_proxy="$1" + export ftp_proxy="$1" +} + +function proxy_unset () +{ + export -n http_proxy + export -n https_proxy + export -n ftp_proxy +} + +unset PROXY_SETUP +function proxy_setup () +{ + [ -n "$PROXY_SETUP" ] && return + case "$OPT_PROXY" in + auto) + proxy_auto + ;; + inherit) + ;; + none) + proxy_unset + ;; + *) + proxy_set "$OPT_PROXY" + ;; + esac + PROXY_SETUP=DONE +} + +function apt-cyg-get-proxy () +{ + proxy_setup + declare -p ftp_proxy http_proxy https_proxy +} + +# Usage: get_pkgname PKGFILE +function get_pkgname () +{ + local tarball="${1##*/}" + echo "$tarball" | sed -re's/-[0-9].*//g' +} + +# PACKAGE_DB is defined at package_db.cc in the cygwin-app setup.exe +# See blow: +# https://www.sourceware.org/cygwin-apps/setup.html +# https://sourceware.org/cgi-bin/cvsweb.cgi/setup/package_db.cc?cvsroot=cygwin-apps +PACKAGE_DB="/etc/setup/installed.db" + +function package_db-version_check () +{ + [ -n "$PACKAGE_DB_VERSION_CHECK_DONE" ] && return + + local vernhdr='INSTALLED\.DB [0-9]+' + local line1; read line1 < "${PACKAGE_DB}" + local dbver=1 + + if [[ "$line1" =~ ^INSTALLED\.DB[[:space:]]+([0-9]+)$ ]]; then + dbver="${BASH_REMATCH[1]}" + else + warning "${PACKAGE_DB} does not have version header. The first line is below:\n" \ + "$(head -n1 "${PACKAGE_DB}")\n" + + # The earlyer version of apt-cyg was not treat version header correctly. + if grep -EHnx "${vernhdr}" "${PACKAGE_DB}" >&2; then + echo "The above line looks like version header, but it is not the first line." >&2 + fi + fi + + if (( dbver < 3 )); then + warning "${PACKAGE_DB} version is less than 3.\n" \ + "Before continuing, recommend to execute below command:\n" \ + " apt-cyg dist-upgrade" + ask_user "Do you continue?" >&2 && { + echo "continue" >&2 + } || { + echo "abort" >&2 + exit 1 + } + elif (( 3 < dbver )); then + error "${PACKAGE_DB} has unknown version header.\n" \ + "Currently apt-cyg supports the DB of ver 3 or ealyer, but your DB is ver $dbver.\n" + ask_user "Do you want to continue at your own risk?" >&2 && { + echo "continue" >&2 + } || { + echo "abort" >&2 + exit 1 + } + fi + + PACKAGE_DB_VERSION_CHECK_DONE=1 +} + +# Usage: package_db-is_registered PKGNAME +function package_db-is_registered () +{ + package_db-version_check + + awk ' + $1 == PKGNAME && NF != 2 {found = 1; exit} + END {exit !found} + ' PKGNAME="$1" "${PACKAGE_DB}" +} + +function package_db-list () +{ + package_db-version_check + + awk ' + NF == 3 { + version = gensub(/^(.*)\.(tgz|tbz|tbz2|tb2|taz|tz|tlz|txz|tar\.(gz|bz2|Z|lz|lzma|xz|zst))$/, "\\1", 1, substr($2, length($1) + 2)); + printf("%s %s %s\n", $1, version, $3); + } + ' "${PACKAGE_DB}" +} + +# Usage: package_db-register PKGFILE USER_PICKED=0 +function package_db-register () +{ + local pkgfile="${1##*/}" + local pkgname="$(get_pkgname "$pkgfile")" + local user_picked="${2:-0}" + local work="/tmp/apt-cyg.$$.${PACKAGE_DB##*/}" + + package_db-version_check + + awk ' + function register() {print PKGNAME " " PKGFILE " " USER_PICKED; registered = 1;} + !registered && PKGNAME < $1 && NF != 2 {register()} + {print $0} + END {if (!registered) register()} + ' PKGNAME="$pkgname" PKGFILE="${pkgfile}" USER_PICKED="${user_picked}" "${PACKAGE_DB}" > "${work}" + + mv "${PACKAGE_DB}" "${PACKAGE_DB}-save" + mv "${work}" "${PACKAGE_DB}" +} + +function package_db_change_mark () # [ ...] +# Change marks of package in PACKAGE_DB. +# means that 0 was automatically installed +# and 1 was manually installed. +{ + local user_picked="$1" + local PACKAGE_NAMES="$(join_str $'\x1c' "${@:2}")" + local work="/tmp/apt-cyg.$$.${PACKAGE_DB##*/}" + + package_db-version_check + + awk -vPACKAGE_NAMES="$PACKAGE_NAMES" -vUSER_PICKED="$user_picked" ' + BEGIN { + split(PACKAGE_NAMES, package_names, "\x1c"); + for(i in package_names) target[package_names[i]]=1; + label = USER_PICKED ? "manually" : "automatically"; + } + NF==2 + NF!=2&&!target[$1] + NF!=2&& target[$1] { + msg = $1 " was " ($3==USER_PICKED?"already ":"") "set to " label " installed."; + $3 = USER_PICKED; + print $0; + print msg > "/dev/stderr" + } + ' "${PACKAGE_DB}" 2>&1 >"${work}" + + mv "${PACKAGE_DB}" "${PACKAGE_DB}-save" + mv "${work}" "${PACKAGE_DB}" +} + + +# Usage: package_db-unregister PKGNAME +function package_db-unregister () +{ + local work="/tmp/apt-cyg.$$.${PACKAGE_DB##*/}" + + package_db-version_check + + awk '!(PKGNAME == $1 && NF != 2) {print $0}' PKGNAME="$1" "${PACKAGE_DB}" > "${work}" + + mv "${PACKAGE_DB}" "${PACKAGE_DB}-save" + mv "${work}" "${PACKAGE_DB}" +} + +# Usage: dep_check DIR ROOTPKGS ... +# Parameters: +# DIR is "depends" or "rdepends". +# ROOTPKGS is root package names to check dependency. +# Return: +# package_name available shallow_depth deep_depth +function dep_check () +{ + awk \ + ' + function min(x,y) {return x < y ? x : y} + function max(x,y) {return x < y ? y : x} + function update_result(dir, rootpkg, pkg, depth, _, i) { + if (0 + result[rootpkg, pkg, "deep"]) { + result[rootpkg, pkg, "shallow"] = min(depth, result[rootpkg, pkg, "shallow"]); + result[rootpkg, pkg, "deep"] = max(depth, result[rootpkg, pkg, "deep"]); + } else { + result[rootpkg, pkg, "deep"] = result[rootpkg, pkg, "shallow"] = depth; + result[rootpkg, pkg, "available"] = 0 + available[pkg]; + result[rootpkg, result[rootpkg, "n"]++] = pkg; + for (i = 0; i < dep[dir, pkg, "n"]; i++) { + update_result(dir, rootpkg, dep[dir, pkg, i], depth + 1); + } + } + } + $1 == "@" { + pkg = $2; + available[pkg] = 1; + } + $1 == "requires:" || $1 == "depends2:" { + for (req = 2; req <= NF; req++) { + reqpkg = gensub(/,$/, "", "g", $req); + dep["rdepends", reqpkg, dep["rdepends", reqpkg, "n"]++] = pkg; + dep["depends" , pkg , dep["depends" , pkg , "n"]++] = reqpkg; + } + } + END { + split(ROOTPKGS, rootpkgs, "\x1c"); + for (k in rootpkgs) { + update_result(DIR, rootpkgs[k], rootpkgs[k], 1); + for(i = 0; i < result[rootpkgs[k], "n"]; i++) { + printf("%-40s %d\t%d\t%d\n", + result[rootpkgs[k], i], + result[rootpkgs[k], result[rootpkgs[k], i], "available"], + result[rootpkgs[k], result[rootpkgs[k], i], "shallow"], + result[rootpkgs[k], result[rootpkgs[k], i], "deep"]); + } + } + } + ' \ + DIR="$1" ROOTPKGS="$(join_str $'\x1c' "${@:2}")" "$(apt-cyg-pathof "setup.ini")" \ + | awk ' + function min(x,y) {return x < y ? x : y} + function max(x,y) {return x < y ? y : x} + { + packages[$1] = pkg = $1; + status[pkg, "available"] = $2; + status[pkg, "shallow" ] = 0 + status[pkg, "shallow"] == 0 ? $3 : min(status[pkg, "shallow"], $3); + status[pkg, "deep" ] = max(0 + status[pkg, "deep"], $4); + } + END { + for (pkg in packages) { + printf("%-40s %d\t%d\t%d\n", + pkg, + status[pkg, "available"], + status[pkg, "shallow"], + status[pkg, "deep"]) + } + } + ' | sort -nrk4 +} + +function apt-cyg-depends () +{ + local pkg + [ -z "$OPT_NO_HEADER" ] && printf "%-40s %s\t%s\t%s\n" "PKGNAME" "AVAIL" "SHALLOW" "DEEP" + dep_check depends "$@" +} + +function apt-cyg-rdepends () +{ + local pkg + [ -z "$OPT_NO_HEADER" ] && printf "%-40s %s\t%s\t%s\n" "PKGNAME" "AVAIL" "SHALLOW" "DEEP" + dep_check rdepends "$@" +} + +function get_module_line () # [] +# Return line numbers of module about begin and end. +{ + grep -nE "^#\s*(BEGIN|END)_MODULE\s*:\s*${1}" "${@:2:1}" | grep -Eo '^[0-9]+' +} + +# EMBED_BEGIN: hhs_utils.bash + +function headtail () # [] +# Split lines from to . +{ + head -n+$2 "${@:3:1}" | tail -n+$1 +} + +function uniqex () +# An alternative uniq command which is not required sort +{ + awk '!c[$0]++' +} + +# EMBED_END: hhs_utils.bash + +function replace_range # [] +{ + awk -vl1=$1 -vl2=$2 -vs="${3//\\/\\\\}" ' + NR < l1 || l2 < NR + !done && l1 <= NR {print s;done=1;} + ' "${@:4:1}" +} + +function replace_module # +{ + local tmpfile; mktmpfn -v tmpfile + local srclines=( $(get_module_line "bash completion for apt-cyg" "$2") ) + local dstlines=( $(get_module_line "bash completion for apt-cyg" "$3") ) + replace_range "${dstlines[@]}" "$(headtail "${srclines[@]}" "$2")" "$3" > "$tmpfile" + mv "$tmpfile" "$3" +} + +function apt-cyg-completion-install () +{ + if [ ! -d "/etc/bash_completion.d" ]; then + error "/etc/bash_completion.d is not exist." + exit 1 + fi + if ! package_db-is_registered "bash-completion"; then + error "bash-completion is not installed." + exit 1 + fi + + local __APT_CYG_SUBCMDS + local __APT_CYG_OPTIONS + local __APT_CYG_SCRIPTPATH="$(realpath "$(type -p apt-cyg)")" + local __APT_CYG_SCRIPTDIR="${__APT_CYG_SCRIPTPATH%/*}" + local __APT_CYG_COMPLETION_DISABLE_AUTOUPDATE="$OPT_COMPLETION_DISABLE_AUTOUPDATE" + + readarray -t __APT_CYG_SUBCMDS < <(grep "^function " "$__APT_CYG_SCRIPTPATH" | awk 'match($2, /apt-cyg-([-_0-9A-Za-z]+)/,m){print m[1]}') + readarray -t __APT_CYG_OPTIONS < <( + awk ' + /^function *parse_args *\(\)/ {proc=1} + /^} *# *\/parse_args( |$)/ {proc=0} + proc && match($0, /^ *(-[^()*]+)\)/, m) { + split(m[1], x, "|"); + for (i in x) print x[i]; + } + ' "$__APT_CYG_SCRIPTPATH" + ) + + cat <<-EOD > /etc/bash_completion.d/apt-cyg + $(declare -p __APT_CYG_{SUBCMDS,OPTIONS,SCRIPTPATH,SCRIPTDIR,COMPLETION_DISABLE_AUTOUPDATE} | sed -E 's/^declare/\0 -g/g') + + # BEGIN_MODULE: bash completion for apt-cyg + # END_MODULE: bash completion for apt-cyg + + complete -o filenames -F __apt-cyg apt-cyg + EOD + + replace_module "bash completion for apt-cyg" "${BASH_SOURCE}" /etc/bash_completion.d/apt-cyg + touch -r "$__APT_CYG_SCRIPTPATH" "/etc/bash_completion.d/apt-cyg" + + echo "A bash completion /etc/bash_completion.d/apt-cyg is installed" +} + +# BEGIN_MODULE: bash completion for apt-cyg + +function __apt-cyg () +{ + local cur prev getsubcmd subcmd + + # Auto update for completion script. + if [ -z "$__APT_CYG_COMPLETION_DISABLE_AUTOUPDATE" -a "$__APT_CYG_SCRIPTPATH" -nt "/etc/bash_completion.d/apt-cyg" ]; then + apt-cyg completion-install >/dev/null 2>&1 + . /etc/bash_completion.d/apt-cyg + __apt-cyg "$@" + return + fi + + _get_comp_words_by_ref -n : cur prev + + getsubcmd=( apt-cyg --completion-get-subcommand $(echo "${COMP_LINE}" | sed -E 's/^[ \t]*[^ \t]+//g;s/[^ \t]+$//g') ) + subcmd="$( "${getsubcmd[@]}" )" + mirroridxopt=( $(echo ${COMP_LINE} | awk 'BEGIN { RS=" " } /^-M|--mirror-index$/ { if(getline) printf "-M %s", $1 }') ) + + case "$subcmd" in + install|depends|rdepends|describe|find|category) + COMPREPLY=( $(awk '/^@ /{print $2}' "$(apt-cyg pathof "${mirroridxopt[@]}" setup.ini)") ) + ;; + remove) + COMPREPLY=( $(apt-cyg --no-header show 2>/dev/null | awk '$0=$1') ) + ;; + pathof) + COMPREPLY=( cache mirror mirrors mirrordir cache/mirrordir setup.ini ) + ;; + *) + COMPREPLY=( "${__APT_CYG_SUBCMDS[@]}" ) + ;; + esac + case "$prev" in + --cache|-c) + readarray -t COMPREPLY < <(compgen -d -- "$cur") + ;; + --mirror|-m) + COMPREPLY=( $(apt-cyg mirrors-list) ) + ;; + --file|-f) + readarray -t COMPREPLY < <(compgen -f -- "$cur") + ;; + --proxy|-p) + COMPREPLY=( auto inherit none http:// ) + ;; + --mirror-index|-M) + COMPREPLY=( $(seq 0 $(($(apt-cyg pathof mirrors | wc -w)-1))) ) + ;; + *) + COMPREPLY+=( "${__APT_CYG_OPTIONS[@]}" ) + ;; + esac + if [ -n "$DEBUG_COMPLETION" ]; then + echo + declare -p BASH_SOURCE "${!COMP@}" getsubcmd subcmd cur prev + echo -n "${PS1@P}" + [[ "$COMP_LINE" =~ ^\ ]] && echo -n "time" # maybe... + echo -n "${COMP_LINE}" + (( ${#COMP_LINE} - COMP_POINT )) && echo -ne "\e[$(( ${#COMP_LINE} - COMP_POINT ))D" + fi + readarray -t COMPREPLY < <(compgen -W "${COMPREPLY[*]@Q}" -- "$cur") + __ltrim_colon_completions "$cur" +} + +# END_MODULE: bash completion for apt-cyg + +function apt-cyg-completion-uninstall () +{ + if [ ! -f /etc/bash_completion.d/apt-cyg ]; then + error "/etc/bash_completion.d/apt-cyg is not exist." + exit 1 + fi + rm /etc/bash_completion.d/apt-cyg + echo "A bash completion /etc/bash_completion.d/apt-cyg is uninstalled" +} + +function apt-cyg-mirrors-list () +{ + setuprc_get_section mirrors-lst | sed -re 's/;.*//g' +} + +function apt-cyg-mirrors-list-long () +{ + setuprc_get_section mirrors-lst | column -t -s ";" +} + +function apt-cyg-mirrors-list-online () +{ + update_cache_for_mirrors_list_online + cat "${apt_cyg_cachedir}/mirrors.lst" +} + +function apt-cyg-benchmark-mirrors () +{ + local mirror result exitcode + local WGET=( wget ) + [ -n "$OPT_BENCHMARK_TIMEOUT" ] && WGET=( timeout "$OPT_BENCHMARK_TIMEOUT" "${WGET[@]}" ) + for mirror; do + result="$( { time "${WGET[@]}" -qO/dev/null -T3 -t 1 "${mirror%/}/$arch/setup.bz2"; } 2>&1 )" + exitcode=$? + if [ $exitcode -ne 0 ];then + if [ $exitcode -eq 124 ]; then + echo -e "Timeout:\t${mirror}" >&2 + else + warning "benchmark failed with wget exitcode $exitcode: $(wget-exitstatus $exitcode): $1" + fi + continue + fi + [[ "$result" =~ real[^0-9]*([0-9]*m[0-9.]*s) ]] && echo -e "${BASH_REMATCH[1]}\t${mirror}" + done +} + +function apt-cyg-benchmark-parallel-mirrors () +{ + local result; mktmpfn -v result + local mirror + for mirror; do echo $mirror; done | lesser-parallel apt-cyg-benchmark-mirrors {} | tee "$result" + echo Finished benchmark. + echo ======================================== + echo Sorted result. + sort -rV "$result" + rm "$result" +} + +function apt-cyg-benchmark-parallel-mirrors-list () +{ + local mirrors + readarray -t mirrors < <(apt-cyg-mirrors-list) + apt-cyg-benchmark-parallel-mirrors "${mirrors[@]}" +} + +function apt-cyg-scriptinfo () +{ + cat<<-EOD + SCRIPT_PATH = "$SCRIPT_PATH" + SCRIPT_FILE = "$SCRIPT_FILE" + SCRIPT_NAME = "$SCRIPT_NAME" + SCRIPT_DIR = "$SCRIPT_DIR" + SCRIPT_REALPATH = "$SCRIPT_REALPATH" + SCRIPT_REALFILE = "$SCRIPT_REALFILE" + SCRIPT_REALNAME = "$SCRIPT_REALNAME" + SCRIPT_REALDIR = "$SCRIPT_REALDIR" + EOD +} + +# Usage: isbusy [file ...] +function isbusy () +{ + perl -e 'foreach $i(@ARGV){if(-f $i){open(DATAFILE,"+<",$i)||exit(0);close(DATAFILE);}}exit(1);' -- "$@" +} + +function apt-cyg-show-packages-busyness () +{ + local pkg lst files + for pkg; do + lst="/etc/setup/${pkg}.lst.gz" + if [ -e "$lst" ]; then + readarray -t files < <(gzip -dc "$lst"|sed 's:^:/:g') + isbusy "${files[@]}" && echo -n "busy: " || echo -n "free: " + echo "$pkg" + fi + done +} + +unset CODEPAGE + +function get_codepage_wt_chcp () +{ + comspec /c chcp.com \ + | awk 'match($NF, /([0-9]+)/, m) { print m[1] == "20127" ? "US-ASCII" : ("CP" m[1]) }' +} + +function get_codepage () +{ + echo -n "${CODEPAGE:=$(get_codepage_wt_chcp)}" +} + +function cp2utf8 () # [ ...] +{ + iconv -f $(get_codepage) -t UTF-8 "$@" +} + +function utf82cp () # [ ...] +{ + iconv -f UTF-8 -t $(get_codepage) "$@" +} + +# dummy command for unknown sum. +# Usage: unknownsum +function unknownsum () +{ + return 1 +} + +# Determine the hash method from a HASH. +# Usage: hash_method HASH +function hash_method () +{ + case "${#1}" in + 32) echo md5 ;; + 40) echo sha1 ;; + 56) echo sha224 ;; + 64) echo sha256 ;; + 96) echo sha384 ;; + 128) echo sha512 ;; + *) echo unknown ;; + esac +} + +# Read md5 and sha{1,224,256,384,512} sums from the FILEs and check them. +# Usage: hash_check [FILE ...] +function hash_check () +{ + local f0 f1 f2 method count=0 + while true; do + while read f0; do + f1="${f0% *}" + f2="${f0#*\*}" + method="$(hash_method "$f1")sum" + if echo "$f0" | "$method" -c --status; then + verbose 0 "hash_check: $method: $f2: OK" + else + verbose 0 "hash_check: $method: $f2: FAILED" + count=$(( count + 1 )) + fi + done 0<${1:-<(cat)} + shift + (( $# <= 0 )) && break + done + if (( 0 < count )); then + verbose 0 "hash_check: WARNING: $count computed checksum did NOT match" + return 1 + fi + return 0 +} + +function kill_all_cygwin_process () +{ + local family + local pid=$BASHPID + local pslog=$(ps -f | tail -n +2 | awk '{print $2, $3}') + readarray -t family < <(echo "$pslog" | awk -v pid=$pid ' + {ppids[$1] = $2;} + END {while (1 < pid) {print "-e\n"pid; pid = ppids[pid];}} + ') + kill -9 $(echo "$pslog" | grep -v "${family[@]}" | awk '{print $1}') ${family[$((${#family[@]} - 1))]} +} + +function apt-cyg-update-setup () +{ + [ -n "$OPT_NO_UPDATE_SETUP" ] && return + local TARGETS=( "${SETUP_EXE}" "${SETUP_EXE}.sig" ) + + pushd . > /dev/null + findworkspace + popd > /dev/null + pushd "$(apt-cyg-pathof cache)" > /dev/null + files_backup "${TARGETS[@]}" + if download_and_verify "https://cygwin.com/${SETUP_EXE}"; then + files_backup_clean "${TARGETS[@]}" + chmod +x "${SETUP_EXE}" + ls -l "${SETUP_EXE}" >&1002 + else + files_restore "${TARGETS[@]}" + error "${SETUP_EXE} could not be downloaded: Rollbacked it and aborted." + exit 1 + fi + + popd > /dev/null +} + +function apt-cyg-setup () +{ + pushd "$(apt-cyg-pathof cache)" > /dev/null + + local setup=( "$PWD/${SETUP_EXE}" -R "$(cygpath -wa /)" "$@" ) + + apt-cyg-update-setup + cd /etc/setup + "${setup[@]}" + + popd > /dev/null +} + +function apt-cyg-dist-upgrade-no-ask () +{ + pushd "$(apt-cyg-pathof cache)" > /dev/null + + local setup=( "$(cygpath -wa "$PWD")\\${SETUP_EXE}" -R "$(cygpath -wa /)" -B -q -n -g) + + apt-cyg-update-setup + cd /etc/setup + cygstart "$COMSPEC" /c 'ECHO Press any key to start dist-upgrade for cygwin '"$arch"' && PAUSE && START /WAIT '"${setup[@]}"' && ash -c "/bin/rebaseall -v" && ECHO dist-upgrade is finished && ECHO Press any key to exit && PAUSE' # ' + kill_all_cygwin_process + + popd > /dev/null +} + +function apt-cyg-dist-upgrade () +{ + echo "Kill all cygwin process and start dist-upgrade." + ask_user "Are you sure ?" && { + echo "Start dist-upgrade ..." + sleep 1 + apt-cyg-dist-upgrade-no-ask + } || { + echo "Abort." + } +} + +function apt-cyg-packages-total-count () +{ + grep ^@ <"$(apt-cyg-pathof setup.ini)" | wc -l +} + +# Usage: apt-cyg-packages-total-size [pattern_of_section] +function apt-cyg-packages-total-size () +{ + local section="^$" + (( 0 < $# )) && section="$1" + + awk -v SECTION="$section" ' + /^@ [ -~]+ *$/ {section = ""} + match($0,/^\[([ -~]*)\] *$/,m) {section = m[1]} + match(section, SECTION) && $1 == "install:" {sum += $3} + END {print sum} + ' "$(apt-cyg-pathof setup.ini)" +} -function usage() +function apt-cyg-packages-cached-count () { - echo apt-cyg: Installs and removes Cygwin packages. - echo " \"apt-cyg install \" to install packages" - echo " \"apt-cyg remove \" to remove packages" - echo " \"apt-cyg update\" to update setup.ini" - echo " \"apt-cyg show\" to show installed packages" - echo " \"apt-cyg find \" to find packages matching patterns" - echo " \"apt-cyg describe \" to describe packages matching patterns" - echo " \"apt-cyg packageof \" to locate parent packages" - echo "Options:" - echo " --mirror, -m : set mirror" - echo " --cache, -c : set cache" - echo " --file, -f : read package names from file" - echo " --noupdate, -u : don't update setup.ini from mirror" - echo " --help" - echo " --version" + pushd "$(apt-cyg-pathof cache/mirrordir)" > /dev/null + find . -type f | grep tar | wc -l + popd > /dev/null } +function apt-cyg-packages-cached-size () +{ + pushd "$(apt-cyg-pathof cache/mirrordir)" > /dev/null + find . -type f -iname '*tar*' -exec ls -l {} + \ + | awk '{sum+=$5}END{print sum}' + popd > /dev/null +} +function apt-cyg-repair-acl () +{ + local target="${1:-/}" + local aclbackup="/tmp/$(date +%Y%m%d_%H%M%S)_acl" + ask_user "$(cat <<-EOD + This subcommand tries to repair the ACL for "${target}". + Maybe it repairs a cygwin that was installed by setup.exe with -B and --no-admin options. + But some package, that are failed to install by the ACL problem, need to be reinstalled. + + And unfortunately, perchance, this might cause some corruptions about the ACL. + You can find a backup of the ACL before being rewritten by this subcommand at below: + "${aclbackup}.bin" + "${aclbackup}.txt" + + Are you sure ? + EOD + )" || exit 1 + echo + + comspec /c icacls.exe "$(cygpath -w "${target}")" /save "$(cygpath -w "${aclbackup}.bin")" > /dev/null + comspec /c icacls.exe "$(cygpath -w "${target}")" | cp2utf8 > "$(cygpath -w "${aclbackup}.txt")" + + comspec /c icacls.exe "$(cygpath -w "${target}")" \ + /grant \ + "%USERDOMAIN%\\%USERNAME%:F" \ + "*S-1-3-1:RX" \ + "Everyone:RX" \ + "CREATOR OWNER:(OI)(CI)(IO)F" \ + "CREATOR GROUP:(OI)(CI)(IO)RX" \ + "Everyone:(OI)(CI)(IO)RX" \ + /remove \ + "NT AUTHORITY\\Authenticated Users" \ + "NT AUTHORITY\\SYSTEM" \ + "BUILTIN\\Administrators" \ + "BUILTIN\\Users" \ + "NULL SID" \ + /inheritance:r \ + | cp2utf8 +} -function version() +function apt-cyg-repair-postinstall () +# Repair postinstall scripts { - echo "apt-cyg version 0.57" - echo "Written by Stephen Jungels" - echo "" - echo "Copyright (c) 2005-9 Stephen Jungels. Released under the GPL." + # Repair type p scripts that were marked "done" incorrectly. + local i done_marked_p=( /etc/postinstall/[0_z]p_*.done ) + [[ ! -e "$done_marked_p" ]] && return + + verbose 0 -e "${SGR_fg_green}${SGR_bold}Repairing:${SGR_reset} type p postinstall scripts that were marked .done incorrectly." + for i in "${done_marked_p[@]}"; do + if [[ "$i" -ot "${i%.done}" ]]; then + rm -v "$i" + else + mv -v "$i" "${i%.done}" + fi + done + echo } +function get_archives_list () #= section type [package_names ...] +#? get archives list by format below: +#? path size digest +#? ... +#? @param section takes section name like curr, prev, test and etc,,, +#? @param type takes type name like install or source. +{ + local section="$1" + local type="$2" + shift 2 + awk -v RS="\n\n@ " -v FS="\n" -v PKGS="$(join_str $'\x1c' "$@")" -v MIRROR="${mirror%/}/" \ + -v SECTION="$section" -v TYPE="$type" ' + BEGIN { + split(PKGS, tmp, "\x1c"); + for (k in tmp) pkgs[tmp[k]] = k; # swap key value + } + { + if(pkgs[$1] == "") { + delete pkgs[$1]; + } else { + section = "curr"; + for (i = 2; i <= NF; i++) { + if (match($i, /^\[(.*)\]$/, m)) { + section = m[1]; + } else if (match($i, TYPE ": *(.*)", m) && section == SECTION) { + result[0+n++]=m[1]; + delete pkgs[$1] + } + } + } + } + END { + for (i =0; i < n; i++) { + print result[i]; + } + if (0 < length(pkgs)) { + printf("\x1b[33;1mWarning:\x1b[0m following packages are not found:")>"/dev/stderr"; + for (pkg in pkgs) printf(" %s", pkg)>"/dev/stderr"; + printf("\n")>"/dev/stderr"; + } + } + ' "$(apt-cyg-pathof "setup.ini")" +} -function findworkspace() +function download_packages () #= pos section type [package_names ...] +#? download packages +#? @param pos takes here or mirror. +#? @param section takes section name like curr, prev, test and etc,,, +#? @param type takes type name like install or source. { - # default working directory and mirror + local pos="$1" + local section="$2" + local type="$3" + shift 3 - mirror=ftp://mirror.mcs.anl.gov/pub/cygwin - cache=/setup - - # work wherever setup worked last, if possible - - if test -e /etc/setup/last-cache - then - tmp="`head -1 /etc/setup/last-cache`" - cache="`cygpath -au "$tmp"`" - fi - - if test -e /etc/setup/last-mirror - then - mirror="`head -1 /etc/setup/last-mirror`" - fi - mirrordir="`echo "$mirror" | sed -e "s/:/%3a/g" -e "s:/:%2f:g"`" + case "$pos" in + here) + ;; + mirror) + cd "$(apt-cyg-pathof cache/mirrordir)" + ;; + *) + error "unknown param: $pos" + exit 1 + ;; + esac - echo Working directory is $cache - echo Mirror is $mirror - mkdir -p "$cache/$mirrordir" - cd "$cache/$mirrordir" + local mirror="$(apt-cyg-pathof mirror)" + local pkgs; readarray -t pkgs < <(get_archives_list "$section" "$type" "$@") + local total="$(printf "%s\n" "${pkgs[@]}" | awk '{sum+=$2}END{print sum;}')" + local n=${#pkgs[@]} + echo "$n packages, total $total bytes will be downloaded." + local line reply=() + for line in "${pkgs[@]}"; do + local tmp=( $line ) + local path="${tmp[0]%/*}/" + local file="${tmp[0]##*/}" + local url="${mirror%/}/${tmp[0]}" + local size="${tmp[1]}" + local digest="${tmp[2]}" + if [ "$pos" = "mirror" ]; then + mkdirp "$path" + pushd "$path" + fi + wget -N "$url" + if ! hash_check <<<"${digest} *${file}" ; then + error "hash did not match: $file" + else + reply+=( "$(cygpath -a "$file" )" ) + fi + [ "$pos" = "mirror" ] && popd + done + REPLY=( "${reply[@]}" ) } +function apt-cyg-source () +{ + download_packages here curr source "$@" +} -function getsetup() +function apt-cyg-mirror-source () { - if test "$noscripts" == "0" -a "$noupdate" == "0" - then - touch setup.ini - mv setup.ini setup.ini-save - wget -N $mirror/setup.bz2 - if test -e setup.bz2 && test $? -eq 0 - then - bunzip2 setup.bz2 - mv setup setup.ini - echo Updated setup.ini - else - wget -N $mirror/setup.ini - if test -e setup.ini && test $? -eq 0 - then - echo Updated setup.ini + download_packages mirror curr source "$@" +} + +function apt-cyg-download () +{ + download_packages here curr install "$@" +} + +function apt-cyg-mirror () +{ + download_packages mirror curr install "$@" +} + +function apt-cyg-browse-homepage-with-mirror-source () # [ ...] +# Browse homepages of packages with mirror-source. +{ + apt-cyg-mirror-source "$@" + + local source sources=( "${REPLY[@]}" ) + + for source in "${sources[@]}"; do + local PN="${source##*/}";PN="${PN%%-[0-9]*}" + local cygport cygports=() + readarray -t cygports < <(tar tf "$source" | grep '\.cygport$') + if (( ${#cygports} <= 0 )); then + readarray -t cygports < <(tar tf "$source" | grep '\.patch$') + fi + for cygport in "${cygports[@]}"; do + local url + read url < <( + tar xf "$source" "$cygport" -O \ + | grep -E '^\+?HOMEPAGE=' \ + | sed -E "s/\+?HOMEPAGE=(.*?)\s*/\1/g;s/[\"']+//g;s/\\$\\{PN\\}/$PN/g" \ + ) + if [[ "$url" =~ ^https?:// ]]; then + echo "Open: $url" + cygstart "$url" else - mv setup.ini-save setup.ini - echo Error updating setup.ini, reverting + error "Invalid URL: $url" fi - fi - fi + done + done } - -function checkpackages() +function apt-cyg-browse-homepage () # [ ...] +# Browse homepages of packages. { - if test "-$packages-" = "--" - then - echo Nothing to do, exiting - exit 0 - fi + local basepath="cygwin.com/packages/summary/" + local pkg summary_path srcsummary_path homepage_url + + pushd "$(apt-cyg-pathof cache)" > /dev/null + for pkg; do + summary_path="${basepath}${pkg}.html" + [ -e "$summary_path" ] || wget -q -N -x "https://$summary_path" + read srcsummary_path < <( + cat "$summary_path" \ + | grep "source package" \ + | sed -E 's@.*\shref="([^"]+)".*@'"${basepath}"'\1@g' + ) + [ -e "$srcsummary_path" ] || wget -q -N -x "https://$srcsummary_path" + read homepage_url < <( + cat "$srcsummary_path" \ + | grep "homepage" \ + | sed -E 's/.*\shref="([^"]+)".*/\1/g' + ) + cygstart "$homepage_url" + done + popd } +function apt-cyg-browse-summary () # [ ...] +# Browse summaries of packages. +{ + local pkg + for pkg; do + cygstart "https://cygwin.com/packages/summary/${pkg}.html" + done +} -# process options +function apt-cyg-listfiles () # [ ...] +# List files 'owned' by package(s). +{ + local lst i sep + for i; do + echo -en "$sep"; sep="\n" + lst="/etc/setup/${i}.lst.gz" + if [ ! -e "$lst" ]; then + echo "apt-cyg-listfiles: package '${i}' is not installed" + continue + fi + zcat "${lst}" | sed -e 's:^:/:g' + done +} -noscripts=0 -noupdate=0 -file="" -dofile=0 -command="" -filepackages="" -packages="" +function apt-cyg-ls-categories () +{ + cat "$(apt-cyg-pathof setup.ini)" \ + | grep category: \ + | sed -r 's/.*: *//g;s/ +/\n/g' \ + | sort \ + | uniq +} -while test $# -gt 0 -do - case "$1" in +function apt-cyg-ls-pkg-with-category () +{ + cat "$(apt-cyg-pathof setup.ini)" \ + | awk '$1=="@"{pkg=$2;}$1=="category:"{for(i=2;i<=NF;i++)print $i,pkg}' \ + | align_columns +} - --mirror|-m) - echo "$2" > /etc/setup/last-mirror - shift ; shift - ;; +function apt-cyg-setuprc-get () #
+{ + setuprc_get_section "$1" +} - --cache|-c) - cygpath -aw "$2" > /etc/setup/last-cache - shift ; shift - ;; +function apt-cyg-set-cache () # [] +{ + (( 0 <$# )) && setuprc_set_section last-cache "$(cygpath -aw "$1")" +} - --noscripts) - noscripts=1 - shift - ;; +function apt-cyg-set-mirror () # [ ...] +{ + set -- "${@%/}" + (( 0 < $# )) && setuprc_set_section last-mirror "${@/%/\/}" + is_official_mirrors_of_cygwin "${@/%/\/}" +} - --noupdate|-u) - noupdate=1 - shift - ;; +function apt-cyg-mark-auto () # [ ...] +{ + package_db_change_mark 0 "$@" +} - --help) - usage - exit 0 - ;; +function apt-cyg-mark-manual () # [ ...] +{ + package_db_change_mark 1 "$@" +} - --version) - version - exit 0 - ;; +function apt-cyg-mark-showauto () +{ + package_db-list | awk '!$3{print $1}' +} - --file|-f) - if ! test "-$2-" = "--" - then - file="$2" - dofile=1 - shift - else - echo 1>&2 No file name provided, ignoring $1 - fi - shift - ;; +function apt-cyg-mark-showmanual () +{ + package_db-list | awk '$3{print $1}' +} - update|show|find|describe|packageof|install|remove) - if test "-$command-" = "--" - then - command=$1 - else - packages="$packages $1" - fi - shift +function apt-cyg-call () # [ [ ...]] +# Call internal function in apt-cyg. +{ + "$@" +} - ;; +function apt-cyg-time () # [ [ ...]] +# Report time consumed to call internal function in apt-cyg. +{ + time "$@" +} - *) - packages="$packages $1" - shift +function apt-cyg-filelist () # [] +# File list like apt-file list +{ + local list_rel_urls + readarray -t list_rel_urls < <( + wget -qO- "https://cygwin.com/packages/summary/${1}.html" \ + | grep -E ']*>list of files' \ + | sed -E 's:.*$/,/^<\/pre>/' | tail -n+2 | head -n-1 \ + | awk -vpkg="$1" '$0=pkg": /"$4' + fi +} - ;; +function apt-cyg-filesearch () # [] +# File search like apt-file search +{ + local cachedir cachefile cachequery columns line package packages url + cachedir="/tmp/.apt-cyg.cache/filesearch" + cachequery="$cachedir/$(sha256sum <<<"$1"|awk '$0=$1')" + mkdirp "$cachedir" + + [ ! -e "$cachequery" ] && cygcheck -p "$1" >"$cachequery" + readarray -t packages < "$cachequery" + packages=( "${packages[@]:1}" ) + for line in "${packages[@]}"; do + columns=( $line ) + url="https://cygwin.com/packages/$arch/${columns[0]%%-[0-9]*}/${columns[0]}" + cachefile="$cachedir/$(mirror_to_mirrordir "$url")" + [ ! -e "$cachefile" ] && wget -qO"$cachefile" "$url" + echo -n "${columns[0]}" + awk '/<\/pre>/{f=0}/
/{f=1}f' "$cachefile" \
+    | grep -E "$1" \
+    | awk '($1=$2=$3="")||1'
+  done
+}
 
-  esac
-done
+PACKAGEOF_CACHE="/tmp/.apt-cyg-packageof.cache.gz"
+function update_packageof_cache ()
+{
+  local i=0 p q path fn
+  local chr=( "=" "-" "/" "|" "\\" )
+  local lstgz_stamp="$(find /etc/setup/ -maxdepth 1 -type f -name '*.lst.gz' -exec stat -c %Y {} + | sort | tail -n1)"
+  local cache_stamp="$(stat "$PACKAGEOF_CACHE" -c %Y 2>/dev/null || echo 0)"
+  local n="$(ls -1 /etc/setup/*.lst.gz|wc -l)"
+  
+  if (( cache_stamp < lstgz_stamp )) || [ -n "$OPT_FORCE_UPDATE_PACKAGEOF_CACHE" ]; then
+    verbose 1 "Updating packageof cache:"
+    progress_init
+    for path in /etc/setup/*.lst.gz; do
+      progress_update $(( i++ )) $n
+      local fn="${path##*/}"
+      zcat "$path" | awk -v PKGNAME="${fn%.lst.gz}" '{print PKGNAME ": " $0;}'
+#      printf "p=%d q=%d i=%d n=%d path=[%s]\n" $p $q $i $n "$path"
+    done | gzip >"$PACKAGEOF_CACHE"
+    progress_finish
+  fi
+}
 
+PROGRESS_CHAR=( "=" "-" "/" "|" "\\" )
+function progress_init ()
+{
+  [ -n "$OPT_NO_PROGRESS" ] && return
+  #          0        1         2         3         4         5
+  #          12345678901234567890123456789012345678901234567890
+  echo -ne "|..................................................|\r" >&1000
+}
 
-if test $dofile = 1
-then
-  if test -f "$file"
-  then
-    filepackages="$filepackages `cat "$file" | awk '{printf "%s ", $0}'`"
+function progress_update () #= current total=100
+{
+  [ -n "$OPT_NO_PROGRESS" ] && return
+  local n=${2:-100}
+  local p=$((2 + 50 * ($1    ) / $n))
+  local q=$((2 + 50 * ($1 + 1) / $n))
+  if (( q - p <= 1 )); then
+    printf "\e[%dG%s" $p "${PROGRESS_CHAR[($1 % 4 + 1) * (1 + $p - $q)]}" >&1000
   else
-    echo File $file not found, skipping
+    printf "\e[%dG%s" $p "$(seq $((q - p))|awk '{printf "="}')" >&1000
   fi
-  packages="$packages $filepackages"
-fi
+}
+
+function progress_finish ()
+{
+  [ -n "$OPT_NO_PROGRESS" ] && return
+  echo >&1000
+}
+
+# Lesser Parallel for Embedding
+# Copyright (c) 2014 Koichi OKADA. All rights reserved.
+# The official repository is:
+# https://github.com/kou1okada/lesser-parallel
+# This script is distributed under the MIT license.
+# http://www.opensource.org/licenses/mit-license.php
+
+LESSER_PARALLEL_MAX_JOBS=${LESSER_PARALLEL_MAX_JOBS:-8}
 
+function lesser-parallel-get-jobs-count ()
+{
+  jobs -l >/dev/null
+  jobs -l | wc -l
+}
 
-case "$command" in
+# Usage: lesser-parallel-restrict-jobs-count MAXJOBS
+function lesser-parallel-restrict-jobs-count ()
+{
+  while [ $(lesser-parallel-get-jobs-count) -ge $1 ]; do
+    sleep 0.2
+  done
+}
 
-  update)
+# Usage: lesser-parallel [command [arguments]] < list_ot_arguments
+function lesser-parallel ()
+{
+  local cmd arg lines line basename ext PARALLEL_SEQ=1
+  readarray -t lines
+  for line in "${lines[@]}"; do
+    basename="$(basename "$line")"
+    ext="${basename##*.}"
+    [ "$ext" = "$basename" ] && ext=""
+    [ "$ext" != "" ] && ext=".$ext"
+    cmd=( )
+    for arg; do
+      case "$arg" in
+      "{}")
+        cmd+=( "$line" )
+        ;;
+      "{.}")
+        cmd+=( "$(basename "$line" "$ext")" )
+        ;;
+      "{/}")
+        cmd+=( "$basename" )
+        ;;
+      "{//}")
+        cmd+=( "$(dirname "$line")" )
+        ;;
+      "{/.}")
+        cmd+=( "$(basename "$basename" "$ext")" )
+        ;;
+      "{#}")
+        cmd+=( "$PARALLEL_SEQ" )
+        ;;
+      *)
+        cmd+=( "$arg" )
+        ;;
+      esac
+    done
+    
+    lesser-parallel-restrict-jobs-count $LESSER_PARALLEL_MAX_JOBS
+    
+    "${cmd[@]}" &
+    PARALLEL_SEQ=$[$PARALLEL_SEQ + 1]
+  done
+  
+  lesser-parallel-restrict-jobs-count 1
+}
 
-    findworkspace
-    getsetup
+#/Lesser Parallel for Embedding
 
-  ;;
+function apt-cyg-help ()
+{
+  usage
+}
 
+# process options
 
-  show)
+arch="${HOSTTYPE:-$(arch)}";arch="${arch/i686/x86}"
+apt_cyg_cachedir="${tmp:-/tmp}/.apt-cyg.cache"
+noscripts=0
+OPT_USER_PICKED=1
+noupdate=0
+OPT_FILES=()
+SUBCOMMAND=""
+ignore_case=""
+force_remove=""
+force_fetch_trustedkeys=""
+no_verify=""
+OPT_PROXY=${APT_CYG_PROXY:-auto}
+OPT_PROXY_REFRESH_INTERVAL=${APT_CYG_PROXY_REFRESH_INTERVAL:-86400} # 86400s = 1day
+OPTS4INHERIT=()
+SETUP_EXE="setup-$arch.exe"
+YES_TO_ALL=false
+INITIAL_ARGS=( "$@" )
+ARGS=()
+
+function parse_args ()
+{
+  local unknown_option END_OPTS
+  
+  while [ $# -gt 0 ]; do
+    case "$1" in
+      
+      --ag)
+        OPT_AG="$1"
+        shift
+        ;;
+      
+      --benchmark-timeout)
+        OPT_BENCHMARK_TIMEOUT="$2"
+        shift 2 || break
+        ;;
+
+      --use-setuprc)
+        warning "Ignore --use-setuprc. This option was removed with issue-24."
+        shift
+        ;;
+      
+      --ignore-case|-i)
+        ignore_case="$1"
+        shift
+        ;;
+      
+      --force-remove)
+        force_remove=1
+        shift
+        ;;
+      
+      --force-fetch-trustedkeys)
+        force_fetch_trustedkeys=1
+        shift
+        ;;
+      
+      --force-update-packageof-cache)
+        OPT_FORCE_UPDATE_PACKAGEOF_CACHE="$1"
+        shift
+        ;;
+      
+      --no-verify|-X)
+        OPTS4INHERIT+=( "$1" )
+        no_verify=1
+        shift
+        ;;
+      
+      --no-check-certificate)
+        OPTS4INHERIT+=( "$1" )
+        WGET+=( "--no-check-certificate" )
+        shift
+        ;;
+      
+      --no-update-setup)
+        OPT_NO_UPDATE_SETUP="$1"
+        shift
+        ;;
+      
+      --no-header)
+        OPT_NO_HEADER="$1"
+        shift
+        ;;
+      
+      --proxy|-p)
+        OPT_PROXY="$2"
+        shift 2 || break
+        ;;
+      
+      --proxy-force-refresh)
+        OPT_PROXY_FORCE_REFRESH="$1"
+        shift
+        ;;
+      
+      --proxy-refresh-interval)
+        OPT_PROXY_REFRESH_INTERVAL="$2"
+        shift 2 || break
+        ;;
+      
+      --completion-get-subcommand)
+        OPT_COMPLETION_GET_SUBCOMMAND="$1"
+        shift
+        ;;
+      
+      --completion-disable-autoupdate)
+        OPT_COMPLETION_DISABLE_AUTOUPDATE="$1"
+        shift
+        ;;
+      
+      --max-jobs|-j)
+        LESSER_PARALLEL_MAX_JOBS="$2"
+        shift 2 || break
+        ;;
+      
+      --mirror|-m)
+        OPT_MIRROR+=( "$2" )
+        shift 2 || break
+        ;;
+      
+      --mirror-index|-M)
+        OPT_MIRROR_INDEX="$2"
+        shift 2 || break
+        ;;
+      
+      --cache|-c)
+        OPT_CACHE="$2"
+        shift 2 || break
+        ;;
+      
+      --noscripts)
+        noscripts=1
+        shift
+        ;;
+      
+      # It will not register the user_picked flag in PACKAGE_DB.
+      # This option is for internal use only.
+      --no-user-picked)
+        OPT_USER_PICKED=0
+        shift
+        ;;
+      
+      --noupdate|-u)
+        noupdate=1
+        shift
+        ;;
+      
+      --ipv4|-4)
+        WGET+=( "--prefer-family=IPv4" )
+        shift
+        ;;
+      
+      --no-progress)
+        OPT_NO_PROGRESS="$1"
+        shift
+        ;;
+      
+      --quiet|-q)
+        OPT_VERBOSE_LEVEL=-1
+        shift
+        ;;
+      
+      --verbose|-v)
+        OPT_VERBOSE_LEVEL=2
+        shift
+        ;;
+      
+      --help)
+        usage
+        exit 0
+        ;;
+      
+      --version)
+        version
+        exit 0
+        ;;
+      
+      --file|-f)
+        [ -n "$2" ] && OPT_FILES+=( "$2" )
+        shift 2 || break
+        ;;
+      
+      --)
+        END_OPTS="$1"
+        shift
+        break
+        ;;
+      
+      -*)
+        unknown_option="$1"
+        break
+        ;;
+      
+      *)
+        if [ -z "$SUBCOMMAND" ]; then
+          SUBCOMMAND="$1"
+        else
+          ARGS+=( "$1" )
+        fi
+        shift
+        
+        ;;
+      
+    esac
+  done
+  [ -n "$END_OPTS" ] && while (( 0 < "$#" )); do ARGS+=( "$1" ); shift; done
+  
+  if [ -n "$OPT_COMPLETION_GET_SUBCOMMAND" ]; then
+    echo "$SUBCOMMAND"
+    exit
+  fi
+  
+  if [ -n "$unknown_option" ]; then
+    error "Unknown option: $unknown_option"
+    exit 1
+  fi
+  
+  if [ $# -gt 0 ]; then
+    error "Number of parameters is not enough: $@"
+    exit 1
+  fi
+  
+} #/parse_args
 
-    echo 1>&2 The following packages are installed:
-    cat /etc/setup/installed.db | awk '/[^ ]+ [^ ]+ 0/ {print $1}'
+parse_args "$@"
 
-  ;;
+: ${OPT_VERBOSE_LEVEL:=$VERBOSE}
+VERBOSE=$OPT_VERBOSE_LEVEL
 
+[ "${#OPT_MIRROR[@]}" -gt 0 ] && apt-cyg-set-mirror "${OPT_MIRROR[@]}"
+[ "${#OPT_CACHE[@]}"  -gt 0 ] && apt-cyg-set-cache  "${OPT_CACHE}"
 
-  find)
+if [ -z "$GPG" -a -z "$no_verify" ]; then
+  error "GnuPG is not installed. Prease install gnupg package or use -X option."
+  exit 1
+fi
 
-    checkpackages
-    findworkspace
-    getsetup
+for file in "${OPT_FILES[@]}"; do
+  if [ -f "$file" ]; then
+    readarray -t -O ${#ARGS[@]} ARGS < "$file"
+  else
+    echo File $file not found, skipping
+  fi
+done
 
-    for pkg in $packages
-    do
-      echo ""
-      echo Searching for installed packages matching $pkg:
-      awk '/[^ ]+ [^ ]+ 0/ {if ($1 ~ query) print $1}' query="$pkg" /etc/setup/installed.db
-      echo ""
-      echo Searching for installable packages matching $pkg:
-      cat setup.ini | awk -v query="$pkg" \
-        'BEGIN{RS="\n\n@ "; FS="\n"; ORS="\n"} {if ($1 ~ query) {print $1}}'
-    done
+update_verbosefor
 
-  ;;
+if [ -n "$OPT_AG" ]; then
+  read AG < <( type -p ag 2>/dev/null )
+  if [ -z "$AG" ]; then
+    warning "ag is not found. ignore option: $OPT_AG"
+    unset OPT_AG
+  fi
+fi
 
 
-  describe)
 
-    checkpackages
-    findworkspace
-    getsetup
-    for pkg in $packages
-    do
-      echo ""
-      cat setup.ini | awk -v query="$pkg" \
-        'BEGIN{RS="\n\n@ "; FS="\n"; ORS="\n"} {if ($1 ~ query) {print $0 "\n"}}'
-    done
+function apt-cyg-update ()
+{
+  findworkspace
+  getsetup
+}
 
-  ;;
 
+function apt-cyg-show ()
+{
+  package_db-version_check
+  [ -z "$OPT_NO_HEADER" ] && echo "The following packages are installed:"
+  package_db-list | awk '($3="")||1' | align_columns
+}
 
-  packageof)
 
-    checkpackages
-    for pkg in $packages
-    do
-      key=`which "$pkg" 2>/dev/null | sed "s:^/::"`
-      if test "-$key-" = "--"
-      then
-        key="$pkg"
-      fi
-      for manifest in /etc/setup/*.lst.gz
-      do
-        found=`cat $manifest | gzip -d | grep -c "$key"`
-        if test $found -gt 0
-        then
-          package=`echo $manifest | sed -e "s:/etc/setup/::" -e "s/.lst.gz//"`
-          echo Found $key in the package $package
-        fi
-      done
-    done
+function apt-cyg-find ()
+{
+  local pkg
+  
+  checkpackages "$@"
+  findworkspace
+  getsetup
+  
+  for pkg do
+    echo ""
+    echo "Searching for installed packages matching $pkg:"
+    package_db-list | awk '$1~query{print $1}' query="$pkg" IGNORECASE="$ignore_case"
+    echo ""
+    echo "Searching for installable packages matching $pkg:"
+    awk -v query="$pkg" -v IGNORECASE="$ignore_case" \
+      'BEGIN{RS="\n\n@ "; FS="\n"; ORS="\n"} {if ($1 ~ query) {print $1}}' \
+      setup.ini
+  done
+}
 
-  ;;
 
+function apt-cyg-category () # 
+#   List all packages in given .
+{
+  apt-cyg-ls-pkg-with-category \
+  | awk -vcategory="$1" '$1==category {print $2}'
+}
 
-  install)
 
-    checkpackages
-    findworkspace
-    getsetup
+function apt-cyg-describe ()
+{
+  local pkg exact
+  
+  checkpackages "$@"
+  findworkspace
+  getsetup
+  for pkg do
+    exact="$(grep -Fx "@ $pkg" setup.ini)"
+    awk -v query="$pkg" -v exact="${exact:+1}" -v IGNORECASE="$ignore_case" \
+      'BEGIN{RS="\n\n@ "; FS="\n"} (exact?$1==query:$1~query){print "@ "$0"\n"}' \
+      setup.ini
+  done
+}
 
-    for pkg in $packages
-    do
+function apt-cyg-packageof ()
+{
+  if [ -z "$OPT_AG" ]; then
+    update_packageof_cache
+    zcat "$PACKAGEOF_CACHE" | grep "$@"
+  else
+    "$AG" -z "$@" /etc/setup/*.lst.gz
+  fi
+}
 
-    already=`grep -c "^$pkg " /etc/setup/installed.db`
-    if test $already -ge 1
-    then
+function apt-cyg-install ()
+{
+  local pkg
+  local script
+  
+  package_db-version_check
+  checkpackages "$@"
+  findworkspace
+  getsetup
+  
+  for pkg do
+    if package_db-is_registered "$pkg"; then
       echo Package $pkg is already installed, skipping
       continue
     fi
-    echo ""
-    echo Installing $pkg
-
+    verbose 0 ""
+    verbose 0 "Installing $pkg"
+    
     # look for package and save desc file
-
-    mkdir -p "release/$pkg"
-    cat setup.ini | awk > "release/$pkg/desc" -v package="$pkg" \
-      'BEGIN{RS="\n\n@ "; FS="\n"} {if ($1 == package) {desc = $0; px++}} \
-       END {if (px == 1 && desc != "") print desc; else print "Package not found"}' 
     
-    desc=`cat "release/$pkg/desc"`
-    if test "-$desc-" = "-Package not found-"
-    then
-      echo Package $pkg not found or ambiguous name, exiting
+    mkdirp "release/$pkg"
+    awk > "release/$pkg/desc" -v package="$pkg" \
+      'BEGIN{RS="\n\n@ "; FS="\n"} {if ($1 == package) {desc = $0; px++}} \
+       END {if (px == 1 && desc != "") print desc; else print "Package not found"}' \
+       setup.ini
+    local desc="$(< "release/$pkg/desc")"
+    if [ "$desc" = "Package not found" ]; then
+      verbose 0 "Package $pkg not found or ambiguous name, exiting"
       rm -r "release/$pkg"
       exit 1
     fi
-    echo Found package $pkg
-
+    verbose 0 "Found package $pkg"
+    
     # download and unpack the bz2 file
-
+    
     # pick the latest version, which comes first
-    install=`cat "release/$pkg/desc" | awk '/^install: / { print $2; exit }'` 
-
-    if test "-$install-" = "--"
-    then
-      echo "Could not find \"install\" in package description: obsolete package?"
+    local install="$(awk '/^install: / { print $2; exit }' "release/$pkg/desc")"
+    
+    if [ -z "$install" ]; then
+      verbose 0 "Could not find \"install\" in package description: obsolete package?"
       exit 1
     fi
-
-    file=`basename $install`
+    
+    local file="$(basename "$install")"
     cd "release/$pkg"
-    wget -nc $mirror/$install
-    
-    # check the md5
-    digest=`cat "desc" | awk '/^install: / { print $4; exit }'` 
-    digactual=`md5sum $file | awk '{print $1}'`
-    if ! test $digest = $digactual
-    then
-      echo MD5 sum did not match, exiting
+    wget -Nc $mirror/$install
+    
+    # check the SHA512 hash
+    local digest="$(awk '/^install: / { print $4; exit }' "desc")"
+    if ! hash_check <<<"${digest} *${file}" ; then
+      verbose 0 "error: hash did not match: $file"
       exit 1
     fi
     
-    echo "Unpacking..."
-    cat $file | bunzip2 | tar > "/etc/setup/$pkg.lst" xvf - -C /
+    verbose 0 "Unpacking..."
+    tar > "/etc/setup/$pkg.lst" xvf "$file" -C / --numeric-owner
     gzip -f "/etc/setup/$pkg.lst"
     cd ../..
     
     
     # update the package database
     
-    cat /etc/setup/installed.db | awk > /tmp/awk.$$ -v pkg="$pkg" -v bz=$file \
-      '{if (ins != 1 && pkg < $1) {print pkg " " bz " 0"; ins=1}; print $0} \
-       END{if (ins != 1) print pkg " " bz " 0"}'
-    mv /etc/setup/installed.db /etc/setup/installed.db-save
-    mv /tmp/awk.$$ /etc/setup/installed.db
+    package_db-register "$file" "$OPT_USER_PICKED"
     
     
     # recursively install required packages
     
-    echo > /tmp/awk.$$ '/^requires: / {s=gensub("(requires: )?([^ ]+) ?", "\\2 ", "g", $0); print s}'
-    requires=`cat "release/$pkg/desc" | awk -f /tmp/awk.$$`
-    
-    warn=0
-    if ! test "-$requires-" = "--"
-    then
-      echo Package $pkg requires the following packages, installing:
-      echo $requires
-      for package in $requires
-      do
-        already=`grep -c "^$package " /etc/setup/installed.db`
-        if test $already -ge 1
-        then
-          echo Package $package is already installed, skipping
+    local requires="$(grep -E "^(requires|depends2): " "release/$pkg/desc" | sed -re 's/^(requires|depends2): *(.*[^ ]) */\2/g' -e 's/,? +/ /g')"
+    
+    local warn=0
+    if [ -n "$requires" ]; then
+      verbose 0 "Package $pkg requires the following packages, installing:"
+      verbose 0 "$requires"
+      for package in $requires; do
+        if package_db-is_registered "$package"; then
+          verbose 0 "Package $package is already installed, skipping"
           continue
         fi
-        apt-cyg --noscripts install $package
-        if ! test $? = 0 ; then warn=1; fi
+        apt-cyg "${OPTS4INHERIT[@]}" --proxy inherit --noscripts --no-user-picked install $package
+        if [ $? -ne 0 ]; then warn=1; fi
       done
     fi
-    if ! test $warn = 0
-    then
-      echo "Warning: some required packages did not install, continuing"
+    if [ $warn -ne 0 ]; then
+      warning "some required packages did not install, continuing"
     fi
     
     # run all postinstall scripts
     
-    pis=`ls /etc/postinstall/*.sh 2>/dev/null | wc -l`
-    if test $pis -gt 0 && ! test $noscripts -eq 1
-    then
-      echo Running postinstall scripts
-      for script in /etc/postinstall/*.sh
-      do
-        $script
-        mv $script $script.done
+    apt-cyg-repair-postinstall
+    
+    local postinstalls=( /etc/postinstall/*.?(da)sh )
+    if [ -e "$postinstalls" -a $noscripts -ne 1 ]; then
+      verbose 0 "Running postinstall scripts"
+      for script in "${postinstalls[@]}"; do
+        $script \
+        && [[ $script != /*/?p_* ]] \
+        && mv $script $script.done
       done
     fi
     
-    echo Package $pkg installed
-
-    done
-
-  ;;
-
-
-  remove)
+    verbose 0 "Package $pkg installed"
+    
+  done
+}
 
-    checkpackages
-    for pkg in $packages
-    do
 
-    already=`grep -c "^$pkg " /etc/setup/installed.db`
-    if test $already = 0
-    then
-      echo Package $pkg is not installed, skipping
+function apt-cyg-remove()
+{
+  local pkg
+  local req
+  
+  package_db-version_check
+  checkpackages "$@"
+  for pkg do
+    if ! package_db-is_registered "$pkg"; then
+      verbose 0 "Package $pkg is not installed, skipping"
       continue
     fi
-
-    dontremove="cygwin coreutils gawk bzip2 tar wget bash"
-    for req in $dontremove
-    do
-      if test "-$pkg-" = "-$req-"
-      then
-        echo apt-cyg cannot remove package $pkg, exiting
+    
+    local dontremove="cygwin coreutils gawk bzip2 tar xz wget bash"
+    for req in $dontremove; do
+      if [ "$pkg" = "$req" ]; then
+        verbose 0 "apt-cyg cannot remove package $pkg, exiting"
         exit 1
       fi
     done
-
-    if ! test -e "/etc/setup/$pkg.lst.gz"
-    then
-      echo Package manifest missing, cannot remove $pkg.  Exiting
+    
+    if [ ! -e "/etc/setup/$pkg.lst.gz" -a -z "$force_remove" ]; then
+      verbose 0 "Package manifest missing, cannot remove $pkg.  Exiting"
       exit 1
     fi
-    echo Removing $pkg
-
+    verbose 0 "Removing $pkg"
+    
     # run preremove scripts
-
-    if test -e "/etc/preremove/$pkg.sh"
-    then
-      "/etc/preremove/$pkg.sh"
-      rm "/etc/preremove/$pkg.sh"
-    fi
-
-    cat "/etc/setup/$pkg.lst.gz" | gzip -d | awk '/[^\/]$/ {print "rm -f \"/" $0 "\""}' | sh
-    rm "/etc/setup/$pkg.lst.gz"
-    rm -f /etc/postinstall/$pkg.sh.done
-    cat /etc/setup/installed.db | awk > /tmp/awk.$$ -v pkg="$pkg" '{if (pkg != $1) print $0}'
-    mv /etc/setup/installed.db /etc/setup/installed.db-save
-    mv /tmp/awk.$$ /etc/setup/installed.db
-    echo Package $pkg removed
-
+    
+    local i postinstalls preremoves
+    readarray -t postinstalls < <(zgrep -E "^etc/postinstall/.*[.](da)?sh$" "/etc/setup/${pkg}.lst.gz" | awk '{print "/"$0}')
+    readarray -t preremoves   < <(zgrep -E "^etc/preremove/.*[.](da)?sh$"   "/etc/setup/${pkg}.lst.gz" | awk '{print "/"$0}')
+    for i in "${preremoves[@]}"; do
+      verbose 0 "Running: ${i}"
+      "${i}"
     done
+    
+    gzip -cd "/etc/setup/$pkg.lst.gz" | awk '/[^\/]$/{printf("/%s\0", $0)}' | xargs -0 rm -f
+    gzip -cd "/etc/setup/$pkg.lst.gz" | awk '/[^./].*[/]$/{printf("/%s\0", $0)}' | sort -zr | xargs -0 rmdir --ignore-fail-on-non-empty
+    rm -f "${postinstalls[@]/%/.done}" "/etc/setup/$pkg.lst.gz"
+    package_db-unregister "$pkg"
+    verbose 0 "Package $pkg removed"
+    
+  done
+}
 
-  ;;
-
-  *)
-
-    usage
-
-  ;;
+function invoke_subcommand ()
+{
+  local SUBCOMMAND="${@:1:1}"
+  local ARGS=( "${@:2}" )
+  local ACTION="apt-cyg-${SUBCOMMAND:-help}"
+  if type "$ACTION" >& /dev/null; then
+    "$ACTION" "${ARGS[@]}"
+  else
+    error "unknown subcommand: $SUBCOMMAND"
+    exit 1
+  fi
+}
 
-esac
+[ -d "$PWD" ] || {
+  warning "Missing current directory: $PWD" \
+          "\nNote that some functions, which require access to current directory, will be failed."
+}
 
+invoke_subcommand "$SUBCOMMAND" "${ARGS[@]}"
diff --git a/known_problems.md b/known_problems.md
new file mode 100644
index 0000000..068d79a
--- /dev/null
+++ b/known_problems.md
@@ -0,0 +1,95 @@
+Known Problems
+--------------
+
+### 2016-02-12: Problems about the ACL
+
+In recent version, the cygwin changed the ACL mechanism .
+So, in the cygwin current version, if it will be installed by `setup.exe` with `-B` or `--no-admin` option,
+the cygwin root (/) does not have correct ACL.
+
+A new subcommand `repair-acl` tries to repair it.
+But some package, that are failed to install by the ACL problem, need to be reinstalled.
+
+
+
+### 2015-04-09: gpgv seems not work correctly on 32 bit environment
+
+To solve this problem, the backend of `verify_signatures` function was changed from `gpgv` to `gpg`.
+thienhv reported this problem [#14](https://github.com/kou1okada/apt-cyg/issues/14). Thanks.
+
+
+
+### 2014-01-17: ca-certificates package is not setup correct at x86_64 with Windows 8.
+
+After clean installing with `setup-x86_64.exe`, there are something wrong about ca-certificate package as below:
+
+    $ ls -lnG \
+         /cygdrive/c/cygwin/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt \
+         /cygdrive/c/cygwin64/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
+    -r--r--r-- 1 1001 314336 Jan 17 18:17 /cygdrive/c/cygwin/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
+    -rw-r--r-- 1 1001      0 Oct 16 12:35 /cygdrive/c/cygwin64/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
+    $ ls -lnG \
+         /cygdrive/c/cygwin/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem \
+         /cygdrive/c/cygwin64/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
+    -r--r--r-- 1 1001 232342 Jan 17 18:17 /cygdrive/c/cygwin/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
+    -rw-r--r-- 1 1001      0 Oct 16 12:35 /cygdrive/c/cygwin64/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
+
+The x86 environment seems no problem except that need an additional setting for wget and ca-certificate. Hmm,,,    
+
+It seems that `/usr/bin/update-ca-trust` is failed at x86_64.
+
+It's ad hoc, but effective way to fix it, is to copy files from cygwin32 to cygwin64, as below:
+
+    cp -a /cygdrive/c/cygwin/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt \
+          /cygdrive/c/cygwin64/etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
+    cp -a /cygdrive/c/cygwin/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem \
+          /cygdrive/c/cygwin64/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
+
+This problem is fixed at 2014-01-24.
+version 0.18.7-1 of p11-kit, p11-kit-trust and libp11-kit0 did not work at the cygwin64 under the Windows 8.
+If you face the above problem, please upgrade these three packages from version 0.18.7-1 to version 0.18.7-2.
+
+For more details, see a thread of below:
+
+* Cygwin mailing list / cygwin / [Re: Is there someone who have a same problem ?](http://cygwin.com/ml/cygwin/2014-01/msg00368.html)
+
+
+
+### 2014-01-15: Signature check failed at cygwinports x86_64.
+
+Oops, setup.bz2 is newer than setup.bz2.sig.
+And it seems to be corrupted.
+
+    $ date
+    Wed Jan 15 01:30:19 JST 2014
+    $ w3m -dump ftp://ftp.cygwinports.org/pub/cygwinports/x86_64/
+    Index of ftp://ftp.cygwinports.org/pub/cygwinports/x86_64/
+    
+    [Upper Directory]
+    md5.sum. . . . . . . Jan 10 14:24    184
+    release/ . . . . . . Jan 10 18:00
+    setup.bz2. . . . . . Jan 10 13:59   579K
+    setup.bz2.sig. . . . Dec  6 11:28     72
+    setup.ini. . . . . . Dec  6 11:28  3.20M
+    setup.ini.sig. . . . Dec  6 11:28     72
+    
+    $ wget -q -O - ftp://ftp.cygwinports.org/pub/cygwinports/x86_64/setup.bz2 | bzip2 -tvv
+      (stdin):
+        [1: huff+mtf rt+rld]
+        [2: huff+mtf data integrity (CRC) error in data
+    
+    You can use the `bzip2recover' program to attempt to recover
+    data from undamaged sections of corrupted files.
+
+As of 2014-01-24, above problem is recovered at least.
+
+
+
+### FIXED: Check setup (`cygcheck -c`) can not detect .tar.xz packages
+
+At cygwin 1.7.25, cygcheck is hardcoded for .tar.gz and .tar.bz2.
+So check setup (`cygcheck -c`) can not detect .tar.xz packages.
+This bug was already fixed with [src/winsup/utils/dump_setup.cc#rev1.28](http://cygwin.com/cgi-bin/cvsweb.cgi/src/winsup/utils/dump_setup.cc?cvsroot=src#rev1.28).
+Please wait a release of cygwin 1.7.26.
+
+This Problem was fixed [cygwin 1.7.26](http://cygwin.com/ml/cygwin-announce/2013-11/msg00027.html) which released at 2013-11-29.
diff --git a/other_forks.md b/other_forks.md
new file mode 100644
index 0000000..e008a22
--- /dev/null
+++ b/other_forks.md
@@ -0,0 +1,42 @@
+Other Known *`apt-cyg`* Forks on the GitHub
+-------------------------------------------
+
+Caution:
+Please do not merge forks that have incompatible licenses.
+
+Ex.) Merging to the GPL from the MIT is possible. But merging to the MIT from the GPL is impossible.
+
+**NB:** For comparison between forks, you can use a fork analyzer tool. Unfortunately, *Stagazers forks* website is down, but you can use [Lovely Forks browser extension](https://github.com/musically-ut/lovely-forks)
+
+### Official (MIT license)
+
+* [transcodes-open / apt-cyg](https://github.com/transcode-open/apt-cyg/network)
+  * [tmshn / cyg-fast](https://github.com/tmshn/cyg-fast/network)
+  * [pi0 / cyg](https://github.com/pi0/cyg/network) (GPLv3)
+
+### Unofficial (MIT license)
+
+* [digitallamb / apt-cyg](https://github.com/digitallamb/apt-cyg/network)
+* [balanx / apt-cyg](https://github.com/balanx/apt-cyg/network)
+* [langlichuan123 / apt-cyg](https://github.com/langlichuan123/apt-cyg/network)
+
+### Unofficial (GPLv2)
+
+* [cfg / apt-cyg](https://github.com/cfg/apt-cyg/network)
+  * [Milly / apt-cyg](https://github.com/Milly/apt-cyg/network)
+* [ashumkin / apt-cyg](https://github.com/ashumkin/apt-cyg/network)
+* [svn2github / apt-cyg](https://github.com/svn2github/apt-cyg/network)
+* [nosuchuser / apt-cyg](https://github.com/nosuchuser/apt-cyg/network)
+* [kazuhisya / apt-cyg64](https://github.com/kazuhisya/apt-cyg64/network)
+* [bnormsoftware / apt-cyg](https://github.com/bnormsoftware/apt-cyg/network)
+* [rcmdnk / apt-cyg](https://github.com/rcmdnk/apt-cyg/network)
+* [buzain / apt-cyg](https://github.com/buzain/apt-cyg/network)
+* [wuyangnju / apt-cyg](https://github.com/wuyangnju/apt-cyg/network)
+* [takuya / apt-cyg](https://github.com/takuya/apt-cyg/network)
+
+### Derivations of non-fork on the github
+
+* [10sr / pac-cyg](https://github.com/10sr/pac-cyg/network)
+* [nobuyo / medy](https://github.com/nobuyo/medy/network)
+* [svnpenn / sage](https://github.com/svnpenn/sage/network) -> [cup / pear](https://github.com/cup/pear/network)
+