Skip to content

Latest commit

 

History

History
1537 lines (1189 loc) · 54.3 KB

readme.org

File metadata and controls

1537 lines (1189 loc) · 54.3 KB
Raw

Litterate macOS configuration

This file might be most readable in an org-mode buffer directly. You can also read it from GitHub but it doesn’t show source block options like :name or :tangle.

See Read me for an introduction about this file, its goal, and how to use it. If you want to note some random “think about it later” notes, please use Roadmap or Transient scrath notes.

Table of contents

Deploy or retrieve configuration

If you need it you can see how to use this file.

(expand-file-name ".")

Pay special attention to any script or installation which would silently modify configuration files: it these changes are needed, replicate them here so they won’t be erased.

In order not to erase any important changes, here is how you should proceed when changing configuration:

  • Make sure no current running processes are likely to modify configuration processes defined in this file.
  • Use the script below to copy actual configuration files into this repository and see what has been changed. Commit them if needed.
  • Then changes everything you want, and generate tangled configuration files so you can see your changes.
  • Finally when you’re satisfied with your new configuration then you can copy files from the repo to $HOME. If you expect some graphical applications to take these changes into accounts, please restart them.
REPO_PATH=<<repo-path()>>
# Files related to Z-shell configuration
cp $HOME/.login  $REPO_PATH/.login
cp $HOME/.zshenv $REPO_PATH/.zshenv
cp $HOME/.zshrc  $REPO_PATH/.zshrc
cp $HOME/.zlogin $REPO_PATH/.zlogin
cp $HOME/.bashrc $REPO_PATH/.bashrc

# Files related to git configuration
cp $HOME/.gitconfig                             $REPO_PATH/.gitconfig
cp $HOME/src/github.com/.gitconfig              $REPO_PATH/src/github.com/.gitconfig
cp $HOME/src/gitlab.com/piotr-yuxuan/.gitconfig $REPO_PATH/src/gitlab.com/piotr-yuxuan/.gitconfig
cp $HOME/src/keybase/.gitconfig                 $REPO_PATH/src/keybase/.gitconfig

# Files related to ssh configuration
cp $HOME/.ssh/config $REPO_PATH/.ssh/config

# Confluent platform custom properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/ksql             && cp $(brew --prefix)/etc/ksql/ksql-server.properties                $REPO_PATH/$(brew --prefix)/etc/ksql/ksql-server.properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/kafka-rest       && cp $(brew --prefix)/etc/kafka-rest/kafka-rest.properties           $REPO_PATH/$(brew --prefix)/etc/kafka-rest/kafka-rest.properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/kafka            && cp $(brew --prefix)/etc/kafka/zookeeper.properties                 $REPO_PATH/$(brew --prefix)/etc/kafka/zookeeper.properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/schema-registry  && cp $(brew --prefix)/etc/schema-registry/schema-registry.properties $REPO_PATH/$(brew --prefix)/etc/schema-registry/schema-registry.properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/kafka            && cp $(brew --prefix)/etc/kafka/kafka.properties                     $REPO_PATH/$(brew --prefix)/etc/kafka/kafka.properties
mkdir -p $REPO_PATH/$(brew --prefix)/etc/connect          && cp $(brew --prefix)/etc/connect/connect.properties                 $REPO_PATH/$(brew --prefix)/etc/connect/connect.properties
REPO_PATH=<<repo-path()>>

# Provoke a reload or $HOME/.login
rm $HOME/.hushlogin

# Copy files from the repo to $HOME.
cp $REPO_PATH/.login  $HOME/.login
cp $REPO_PATH/.zshenv $HOME/.zshenv
cp $REPO_PATH/.zshrc  $HOME/.zshrc
cp $REPO_PATH/.zlogin $HOME/.zlogin
cp $REPO_PATH/.bashrc $HOME/.bashrc

# Files related to git configuration
cp $REPO_PATH/.gitconfig                             $HOME/.gitconfig
cp $REPO_PATH/src/github.com/.gitconfig              $HOME/src/github.com/.gitconfig
cp $REPO_PATH/src/gitlab.com/piotr-yuxuan/.gitconfig $HOME/src/gitlab.com/piotr-yuxuan/.gitconfig
cp $REPO_PATH/src/keybase/.gitconfig                 $HOME/src/keybase/.gitconfig

# Files related to ssh configuration
cp $REPO_PATH/.ssh/config $HOME/.ssh/config

# Confluent platform custom properties
cp $REPO_PATH/$(brew --prefix)/etc/ksql/ksql-server.properties                $(brew --prefix)/etc/ksql/ksql-server.properties
cp $REPO_PATH/$(brew --prefix)/etc/kafka-rest/kafka-rest.properties           $(brew --prefix)/etc/kafka-rest/kafka-rest.properties
cp $REPO_PATH/$(brew --prefix)/etc/kafka/zookeeper.properties                 $(brew --prefix)/etc/kafka/zookeeper.properties
cp $REPO_PATH/$(brew --prefix)/etc/schema-registry/schema-registry.properties $(brew --prefix)/etc/schema-registry/schema-registry.properties
cp $REPO_PATH/$(brew --prefix)/etc/kafka/kafka.properties                     $(brew --prefix)/etc/kafka/kafka.properties
cp $REPO_PATH/$(brew --prefix)/etc/connect/connect.properties                 $(brew --prefix)/etc/connect/connect.properties

Prioritary domains

Of course evaluation order matters. For example, secrets must be loaded before they are used.

Hello

Simple function providing a warm greeting towards the user.

function hello {
  echo "Hello, $USER!"
}

Managed configuration files

Here we set the header for configuration files managed from here.

We set Z-shell startup files in the same order Z-shell run them when it starts up. No previous source blocks should be tangled to these files because they would appear before the file header then, and before the shebang line.

Each of these Z-shell files is reset by Babel when tangled. As they’re to be executed by Z-shell, they start with the appropriate shabang. The first one of them is not standard in macOS, I describe it in a later section.

#!/usr/bin/env zsh
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.login is run when a user logs in and when no .hushlogin is
#   present.

The next three files are specific to Z-shell which I choose to use everywhere. Here they are presented is the order they are looked up.

#!/usr/bin/env zsh
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.zshenv is first conf file. It's run for all shells.
#!/usr/bin/env zsh
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.zshrc is run for all interactive shells, that's to say any
#   shell I can write and send commands to.
#!/usr/bin/env zsh
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.zlogin is run for all login shells, that's to say any shell
#   started as a fundamental terminal emulator interpretor (wording is
#   approximative).
#!/usr/bin/env bash
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.bashrc is a bash configuration file unrelated to zsh. It's
#   kept here because some ancillary sh scripts need some
#   configuration too.

Now we define configuration files related to git.

# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.gitignore contains user-wide git
#   configuration. Configuration specific to a repository or
#   git-repository manager such as GitHub or GitLab are to be found in
#   dedicated configuration files.
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/src/github.com/.gitconfig contains git configuration related
#   to git-repository manager GitHub. Configuration specific to a
#   given repository $REPO is to be found in dedicated configuration
#   files $REPO/.git/config.
# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/src/gitlab.com/piotr-yuxuan/.gitconfig contains git
#   configuration related to git-repository manager
#   GitHub. Configuration specific to a given repository $REPO is to
#   be found in dedicated configuration files $REPO/.git/config.
# - This file has been written automatically from configuration
#   repository /Users/p2b/environment-configuration.
#   You can author this file directly, or update the script.
# - $HOME/src/keybase/.gitconfig contains git configuration related to
#   repositories hosted on keybase. Configuration specific to a given
#   repository $REPO is to be found in dedicated configuration files
#   $REPO/.git/config.

The next header documents configuration file related to ssh

# - This file has been written automatically from configuration
#   repository <<repo-path()>>.
#   You can author this file directly, or update the script.
# - $HOME/.ssh/config is used to configure general ssh behaviour. For
#   specific configuration related to git, see the related section in
#   the configuration repository.

Locales

export LC_ALL=en_US.UTF-8

Brew path

path=('/usr/local/bin' '/usr/local/sbin' $path); export PATH
brew --prefix

Secrets in the command line

Sensitive environment variables shouldn’t be handled as shareable configuration but as secrets. In a nutshell it merely uses gpg:

SECRET_FILE=$(gmktemp)
CONFIGURATION_SECRET_KEY='<< my gpg key id >>'
echo "echo 'Secrets loaded'" > ${SECRETS_FILE}

# Encrypt your secrets
gpg --output ${SECRETS_FILE}.asc --encrypt --recipient ${CONFIGURATION_SECRET_KEY} ${SECRETS_FILE}; rm ${SECRETS_FILE}

# Source your secrets with ZSH process substitution
source <(gpg --quiet --decrypt --recipient ${CONFIGURATION_SECRET_KEY} ${SECRETS_FILE}.asc)

# Decrypt your secrets
gpg --decrypt --recipient --output ${SECRETS_FILE} ${CONFIGURATION_SECRET_KEY} ${SECRETS_FILE}.asc; rm ${SECRETS_FILE}.asc

I want to enforce some assumptions on existing environment variables.

function expect-configured-environment-variable () {
    VAR_NAME=$1
    if [[ -z "${(P)VAR_NAME}" ]]; then
        echo "Configuration error: expect environment variable $VAR_NAME to be set"
    fi
}
source <(gpg --quiet --decrypt --recipient $(cat <<repo-path()>>/.gpg-id) <<repo-path()>>/.zsh-secrets.asc)

Flat domains

Better cat and ls

brew install ls exa
alias ls=exa
alias cat='bat --paging never'

Goals

  1. (Trust) Rely on Emacs.<<goal-0>>
  2. (KISS principle) I want my configuration to be kept simple, stupid: most systems work better when kept simple. No need to change mindset, no need to burden beginners with something abstruse. <<goal-1>>
  3. (principle of locality) I want a self-contained configuration in one single place. I want different settings related to the same stuff be close to each other. No other, hidden data are needed for the understanding and nobody else should produce side-effect. <<goal-2>>
  4. (predictability) I want my command-line environment to keep the same behaviour, no matter how many times I invoke and execute this configuration file. Also, I want to be able to come back to a previous behaviour when someting goes wrong. <<goal-3>>
  5. (declarativeness) I want to keep track of the reasonning which led to decisions and choices about this configuration. <<goal-4>>

Non goal: making this file portable and installable over the network on new computers.

  • It would be rather useless: I usually work on one computer everyday and don’t change that often.
  • It would contradict first principle. This is not an automatic installation script, write one if you want so.

Choices made to reach these goals

  1. Rely on Emacs
    • why LISP?
    • why Emacs?
    • Emacs is a most awesome piece of software written in a most powerful programming language. I don’t see any trouble to depend on it. On the contrary, I want Emacs to slowly infuse into my mind and to shape my thoughts.
  2. KISS principle
    • Don’t modify actual configuration without a conscious action from the user.
    • User doesn’t need to be conscious about this file to modify their system configuration, they can always catch up later and see what’s changed.
    • This is only a documentation, not an automatic deployment system or a configuration manager. It is what you want to do and why you chose that, no how to make it happen.
  3. Locality
    • Only use this repository for most of your general environment configuration. Structure decision by domains so different settings about the same domain end near each other. If need be, link across related concerns.
    • Specialised settings related to something specific (Emacs, IntelliJ) live in their own repositories, they are imported as git modules.
    • Any shell setting must originate from here. I’ve chosen to build Z-shell with option –without-etcdir so /etc/z* become muted and this goal is met (debatable, perhaps it’s less KISS).
  4. Predictability
    • Use git to version this repository so you can always checkout a prior state.
  5. Declarativeness
    • Use git to keep track of changes in time. You can see differences from the last commit as well as use git bissect when you don’t understand something weird.
    • Use litterate programming to explain all the whereabouts about the configuration you choose to set up.

Learnt mistakes

$HOME as a git repository for configuration files

Why it’s not a good idea:

  • Quite a lot of programs behave differently when they are in a git repository.
  • It’s tedious to gitignore everything then un-ignore only specific files. Last time I checked I ended up with long, inexplicable .gitignore.
  • Do you feel completely quiet doing git bisect or git reset --hard on your $HOME? Why not trying git clean -x in a deep subdirectory whilst you think it’s a repository but it isn’t?

Each shell functions has its own $HOME/bin file

Why it’s not a good idea:

  • Is there any advantage doing that way in the context of a shell?
  • It’s a bad idea because it strongly separates the function from where it’s used.
  • It can help create boring rookie bugs like keychain fork bomb.

(macOS) Use launchctl setenv for environment variables

Why it’s not a good idea:

  • Most GUI apps don’t refresh their environment once they’re started.
  • launchtl define the environment for all macOS applications (including the terminal emulator) and then you append additional variables in your shell startup files. It’s pretty useless to expose cli variables to graphical applications.
  • I don’t find that kind of stuff actually fun. The proprietary Apple environment API aren’t really stable to my eyes, I don’t feel amused to waste time about them so I prefer to stay the most Unix-like possible.

If you need to tune the environment of an app, you can use plist key LSEnvironment (but leave a comment here so you will remind this in 8 months). Under the exceptional case you would need something really user-specific, you could indeed use launchctl setenv in $HOME/.login.

Use magic tools or nuclear bombs to kill a mosquito

What do you think of GitFlow? I think the basic feature branching naming is a good idea but everything else is complicated and cluttered. I always need to read a translation to raw git commands to feel confident about what’s going on. It’s a complete framework to put in your head when basic git commands are much more simple to reason about.

Perhaps you want something awfully difficult and complex so it looks more professional. Automate dotfiles deployments with Puppet or Chef if you go for it.

The ease of use is not interesting in front of the simplicity of use. Don’t automate overly, don’t add too much incidental complexity.

Put a lot of thing into $HOME/.login to fasten shell startup

Only put here side-effects which are to be executed once at start-up. If you want your shell to take less time to boot, choose carefully which Z-shell plugins you launch. Keep in mind that this file isn’t standard in macOS.

Spread related settings

I previously modified /etc/* and six months later I loose one full hour because of an unwise side-effect. As a result, I got the strong opinion that no side-effect should be defined out of this file. Any code, data or settings which are related one to another should be physically close or in the same location. To ensure so I compile Z-shell with option --without-etcdir <<–without-etcdir>> so it doesn’t read rc files in /etc.

How to use this file

This document merely discusses the choices made about the configuration and presents the resulting configuration files. How to bind this result to any useful configuration files is a choice left to be made to the user.

The inner repository tree structure of the tangled files matches the one of the user $HOME. That’s to say, the tangled file $REPO_PATH/.emacs.d/init.el mirrors $HOME/.emacs.d/init.el. Only exceptions: readme.org (and others like readme.pdf, etc.) and .git repository folder.

It is very very straightforward to use this file:

  • Evaluate code snippets with org-babel-execute-src-block.
  • Generate tangled files with org-babel-tangle.

Then you basically have three options:

  • Create symbolic links manaually
  • Rely on GNU Strow to manage symbolic links
  • Copy files from this repository onto the actual configuration files

The most simple, stupid options is to use cp so actual conf files modification requires an explicit action from the user.

Included in this file is a script to <a href=”copy files from $HOME to the repo”>copy files from $HOME to the repo. It’s useful when you want see with git diff what is about to change. Let’s define the path of this repository so we can use it later with reference extension.

You can look for read me sections for an introduction about this file, its goal, and how to use it.

Quick introduction on org-mode and Babel

Thorough descriptions of org-mode and Babel are available on their respective manual.

In a very simple way, this file contains source blocks. Emacs, when asked properly (thanks to Babel), can write these blocks out to specified files. For example, the following block would be written out to ./my-file.txt – this file is said to be tangled.

#+BEGIN_SRC text :tangle ./my-file.txt
  This is the new content of file
#+END_SRC

You can also execute code in source blocks like the following one. Here I specify shell as the source language so it will get executed ith my default shell (which is Z-shell).

#+BEGIN_SRC shell :results silent
  cp ./my-file.txt ./renamed-file.txt
#+END_SRC

Finally, let’s use block arguments and reference expension. Arguments are passed to the source block as they would be set in the language. Reference expension are a bit like macro and get replaced in the body of the block before it gets evaluated. This last block is equivalent to the previous one.

#+NAME: current-file
#+BEGIN_SRC emacs-lisp
  (concat "my-file" ".txt")
#+END_SRC

#+BEGIN_SRC shell :results silent :var CURRENT_PATH=(expand-file-name ".")
  cp ./<<current-file()>> $CURRENT_PATH/renamed-file.txt
#+END_SRC

Yeah, I know, Emacs is powerful and can turn a litterate description of your settings into a effectful REPL.

Git, the version control software

expect-configured-environment-variable GITHUB_TOKEN_CLI
[includeIf "gitdir:~/src/github.com/**"]
    path = ~/src/github.com/.gitconfig
[includeIf "gitdir:~/src/gitlab.com/piotr-yuxuan/**"]
    path = ~/src/gitlab.com/piotr-yuxuan/.gitconfig
[includeIf "gitdir:~/src/keybase/**"]
    path = ~/src/keybase/.gitconfig

[gpg]
    program = /usr/local/bin/gpg
[commit]
    gpgsign = true
[gc "refs/remotes/*"]
    reflogExpire = 35 days
    reflogExpireUnreachable = 35 days
[core]
    excludesfile = ~/.gitignore






Tags in this file


Read me


Sections with this tag describe this file and how to get acquainted
  with it. Click on the tag to see all sections which bear it.




Text or code editor




Programming language




This current local machine




macOS configuration




Shell, or terminal configuration




Small, minor tool which can help




Day to day tasks to keep your system up to date : bump versions,
  update declarative versions, synchronise settings.


Organisation of $HOME


/Users/$USER
├── .emacs.d/
│   ├── init.el (generated by init.org)
│   └── init.org (litterate init.el)
├── Desktop
├── bin/ (binary files or executable scripts)
├── dist/ -> $HOME/.m2/repository/
├── environment-configuration/
│   └── readme.org (this file)
├── img/ -> Pictures/
│   ├── screenshots/ (where screenshots are put)
│   └── pvt -> $HOME/pvt/img/
├── lib/ -> Library/
├── man/c
│   └── pvt -> $HOME/pvt/man/
├── mov/ -> Movies/
│   └── pvt -> $HOME/pvt/mov/
├── net/ -> Downloads/
│   └── pvt -> $HOME/pvt/net/
├── pkg/
├── pvt -> Documents
│   ├── img/
│   │   └── library -> $HOME/pvt/library/
│   ├── library/ (enclosed library files, flat directory)
│   ├── man/
│   ├── mov/
│   │   └── library -> $HOME/pvt/library/
│   ├── net/
│   └── snd/
│       ├── music/ (flat directory or music I listen to)
│       └── library -> $HOME/pvt/library/
├── snd -> Music/
│   └── pvt -> $HOME/pvt/snd
└── src/ (for source code)
    ├── github.com/ (host/username/repo)
    └── …



I feel like it would really be a terrible idea to actually rename user
  folders like Documents and Movies because there are very standard
  folders which are not meant to change. Even with a standard macOS tool
  to say “OK, now let’s change the default folder for pictures from
  Pictures to img”, I can’t guarantee that no program wouldn’t blindly
  assume it exists.


I have chosen to hide default folders and create symbolic links to
  them so it looks like what I want but the change doesn’t bring too
  deep implication and weird bugs.


These file names are inspired from what golang expects.


$HOME/.login


$HOME/.login isn’t a standard file in macOS. However, I’ve found
  it a nice tool for side-effects which must be invoked once in a
  while and not for each new shell (for example ssh configuration).


Custom shell prompts when you first open a terminal in a session. You
  wanna trigger it only once after your login or $HOME.login is
  reloaded.


# If it's in a terminal and no .hushlogin is present
if [[ ( -t 1 ) && ( ! -f $HOME/.hushlogin ) ]]; then

    source $HOME/.login
    touch $HOME/.hushlogin # Don't execute it again

    archey 2>& /dev/null
    echo "$(fortune)\n"
fi


Also, you still need to remove hushlogin on startup.


Important : this file can’t contain variable like $USER, change it to
  what you need to explicit absolute path.


;; $HOME/Library/LaunchAgents/user.login.plist


<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
	  <dict>
		  <key>Label</key>
		  <string>user.login</string>

		  <key>ProgramArguments</key>
		  <array>
			  <string>/bin/rm</string>
			  <string>$USER/.hushlogin</string> <!-- FIXME -->
		  </array>

		  <key>RunAtLoad</key>
		  <true/>

		  <key>WatchPaths</key>
		  <array>
			  <string>$HOME/.login</string>
		  </array>
	  </dict>
</plist>


This file is not included in the source block which copy files to
  $HOME because it is almost never updated. In addition to that, it must
  be reloaded with the following script whenever it changes.


launchctl unload -w $HOME/Library/LaunchAgents/user.login.plist
launchctl load -w $HOME/Library/LaunchAgents/user.login.plist


How to set $PATH


How to set $PATH in Z-shell https://stackoverflow.com/a/18077919.


Here I choose to set $PATH within context. That’s to say, when I need
  to append something to $PATH because I’m setting up a program, I do it
  in the context of this program. I always do it from this file. As a
  result, I can always easily find where $PATH is set (it originates
  from this file) and why it’s been set up this way.


I have previously gone great length to use path_helper. I found its
  behaviour to be hard to predict so I even implemented it again in a
  simple zsh script which read .path as well as .path.d/*. I’m no
  longer amused with that kind of accidental complexity and now I prefer
  to set path here in this file so I know the context.


Hostname


https://apple.stackexchange.com/a/90873


sudo scutil --set ComputerName $hostname
sudo scutil --set LocalHostName $hostname
sudo scutil --set HostName $hostname

dscacheutil -flushcache
# Perhaps you will need to restart applications or even you computer
# for this setting to take effect.


Brew, macOS package manager


expect-configured-environment-variable HOMEBREW_GITHUB_API_TOKEN


Keeping your system up to date.


After this page. It would be wonderful to use mas. Err, wait, not
  really. I use almost no program from app store except Line. If I use
  Line, I must be sure I can avoid OS upgrades.


When running this script with greedy cask update, better to run this
  snippet in the CLI as your password might be asked for.


brew update
brew upgrade
brew cask upgrade --greedy # greedy reinstall latest version of
			   # unversioned casks. Suboptimised, takes
			   # significantly more time.
brew cleanup -s
brew cask cleanup
serialize_brew_state
echo "you should commit your environment configuration"


Reproducible environment


How to get a reproducible configuration with brew? I will later use
  Nix, the functional package manager. For now I can at least get a
  description of installed software with brew.


function serialize_brew_state {
  REPO_PATH=<<repo-path()>>

  # We don't care about actual Brewfile because it will be regenerated
  # soon and it's versionned anyway.
  rm $REPO_PATH/Brewfile

  # Generate Brewfile with hard location
  pushd $REPO_PATH
  BREWFILE_PATH=. brew bundle dump
  popd

  # List installed software with version
  brew list --versions > $REPO_PATH/brew-list-versions

  # List installed software with version (cask)
  brew cask list --versions > $REPO_PATH/brew-cask-list-versions
}


Change current directory to frontmost window of macOS Finder.


pfd () {
    currFolderPath=$( /usr/bin/osascript <<EOT
	  tell application "Finder"
	      try
	  set currFolder to (folder of the front window as alias)
	      on error
	  set currFolder to (path to desktop folder as alias)
	      end try
	      POSIX path of currFolder
	  end tell
EOT
    )
    "$currFolderPath"
}


Recursively delete .DS_Store files


alias cleanupDS="find . -type f -name '*.DS_Store' -ls -delete"


Hide or show hidden files in Finder


alias finderShowHidden='defaults write com.apple.finder AppleShowAllFiles TRUE'
alias finderHideHidden='defaults write com.apple.finder AppleShowAllFiles FALSE'


Compile Z-shell files (zcompile)


It looks tempting to compile Z-shell files in an attempt to boost your start up time.


https://github.com/antonio/zsh-config/blob/285f5ac6955c19908c30935b312d3521d0e0b5e2/help/zcompile


However it appears not to be a good idea because whenever a script
  silently try to append anything into a Z-shell configuration files
  (for example, an installation script add something in $PATH) it
  creates a new file which has precedence over compiled file (because of
  a more recent timestamp).


Keep it simple: don’t use compile startup files so you can modify your
  Z-shell files without much thinking about that.


Network helpers


alias myip='curl ip.appspot.com'                    # myip:         Public facing IP Address
alias netCons='lsof -i'                             # netCons:      Show all open TCP/IP sockets
alias flushDNS='dscacheutil -flushcache'            # flushDNS:     Flush out the DNS Cache
alias lsock='sudo /usr/sbin/lsof -i -P'             # lsock:        Display open sockets
alias lsockU='sudo /usr/sbin/lsof -nP | grep UDP'   # lsockU:       Display only open UDP sockets
alias lsockT='sudo /usr/sbin/lsof -nP | grep TCP'   # lsockT:       Display only open TCP sockets
alias ipInfo0='ipconfig getpacket en0'              # ipInfo0:      Get info on connections for en0
alias ipInfo1='ipconfig getpacket en1'              # ipInfo1:      Get info on connections for en1
alias openPorts='sudo lsof -i | grep LISTEN'        # openPorts:    All listening connections
alias showBlocked='sudo ipfw list'                  # showBlocked:  All ipfw rules inc/ blocked IPs


Syntactic sugar for search


alias f="find . -name "                     # f:        Quickly search for file
ff () { /usr/bin/find . -name "$@" 2>/dev/null ; }      # ff:       Find file under the current directory
ffs () { /usr/bin/find . -name "$@"'*' 2>/dev/null ; }  # ffs:      Find file whose name starts with a given string
ffe () { /usr/bin/find . -name '*'"$@" 2>/dev/null ; }  # ffe:      Find file whose name ends with a given string


iTerm2


I use iTerm2 as a terminal but you could use built-in Apple
  Terminal. I use Z-shell as a shell. Recently I’ve been trying
  extraterm.


Installation


brew cask info iterm2-nightly


Configuration


Configuration can be exported into json.


Integration


path+=("$HOME/.iTerm2"); export PATH


Z-shell


Installation


See Brewfile for compilation switch. –without-etcdir is enabled: it’s
  a custom setting so you have to know it but it removes even the
  possibility that data in etc dir change the behaviour of zsh, which
  can be quite painful to debug.


brew install zsh --without-etcdir --with-unicode9 --with-pcre


Define Z-shell as you default session shell:


chsh -s $(which zsh)


I install the latest (stable) version of Z-shell. I compile it with an
  option which explicitly disables the reading of Zsh rc files in /etc
  <<without-etcdir>>, so it further enforces that all configuration must
  come from here.


TODO zsh opts?


Basic configuration


This obviously targets interactive shells.


export VISUAL='emacsclient --create-frame'
export EDITOR='emacsclient'
export ARCHFLAGS="-arch x86_64"


I need history variables for interactive shells, they would
  be useless elsewhere.


export HISTSIZE=10000000
export HISTFILESIZE=100000000
export HISTTIMEFORMAT="%d/%m/%y %T "

# https://leetschau.github.io/remove-duplicate-zsh-history.html
setopt EXTENDED_HISTORY          # Write the history file in the ":start:elapsed;command" format.
setopt INC_APPEND_HISTORY        # Write to the history file immediately, not when the shell exits.
setopt SHARE_HISTORY             # Share history between all sessions.
setopt HIST_EXPIRE_DUPS_FIRST    # Expire duplicate entries first when trimming history.
setopt HIST_IGNORE_DUPS          # Don't record an entry that was just recorded again.
setopt HIST_IGNORE_ALL_DUPS      # Delete old recorded entry if new entry is a duplicate.
setopt HIST_FIND_NO_DUPS         # Do not display a line previously found.
setopt HIST_IGNORE_SPACE         # Don't record an entry starting with a space.
setopt HIST_SAVE_NO_DUPS         # Don't write duplicate entries in the history file.
setopt HIST_REDUCE_BLANKS        # Remove superfluous blanks before recording entry.


Oh-my-zsh


I would clearly prefer Z-plug here. I will use it when I’ll have
  time.


 export ZSH=$HOME/.oh-my-zsh

 # I should better use TERM_PROGRAM
 case $TERM in
     # - Emacs term and multi-term
     eterm-color) export ZSH_THEME=lambda
		   ;;
     # - Emacs eshell
	# not a zsh shell, don't read this file
     # - iTerm
     # - Terminal (macOS standard app set up for this)
     xterm-256color) export BULLETTRAIN_DIR_EXTENDED=2
		      export BULLETTRAIN_PROMPT_ADD_NEWLINE=false
		      export BULLETTRAIN_PROMPT_ORDER=(git context dir time)
		      export ZSH_THEME=bullet-train
		      ;;
 esac

 # Yes, I'm a sinner. (enables oh-my-zsh auto updates)
 export DISABLE_UPDATE_PROMPT=false

 # zsh-autosuggestions git git-extras emacs aws npm node go golang lein
 plugins=(git git-extras emacs go golang lein zsh-iterm-touchbar)
 source $ZSH/oh-my-zsh.sh

 if [[ 'iTerm.app' = $TERM_PROGRAM ]]; then
   source "${HOME}/.iterm2_shell_integration.zsh"
 fi


Firefox dev edition


I’ve modified chrome/userChrome.css to remove the “red dot” on new tab
  title change, which Firefox makes a less aggressive blue
  light. Actually, even if not that aggressive I don’t want to be
  disturbed by Twitter and Facebook.


.tabbrowser-tab > .tab-stack > .tab-content[pinned][titlechanged] {
  background-image: none !important;
}


The idea comes from this answer.


SSH configuration


Only express here configuration related to ssh connection to some
  actual host through ssh. For git ssh keys, see the git config which
  gracefully handles multiplie identities.


# Add these keys once and for all
ssh-add -K $HOME/.ssh/mine.pem


Host *
  UseKeychain           yes
  AddKeysToAgent        yes

Host github.com
  HostName              github.com
  User                  git

Host gitlab.com
  HostName              gitlab.com
  User                  git


Shell utilities


GPG Terminal


export GPG_TTY=$(tty)


Jump over directories


I use z for it. It’s like j but if I remember well it’s in pure
  script.


source <<brew-prefix()>>/etc/profile.d/z.sh


I also define some shortcuts. They are to be used only by myself
  directly from the command line, hence I put them in ~./.zshrc.


export SRC="$HOME/src"
export GH="$SRC/github.com"
export GL="$SRC/gitlab.com"
export SND="$HOME/snd"
export PVT="$HOME/pvt"
export MOV="$HOME/mov"
export NET="$HOME/net"


Colourful man pages


This come from boredzo.org.


man() {
    env \
	  LESS_TERMCAP_mb=$(printf "\e[1;31m") \
	  LESS_TERMCAP_md=$(printf "\e[1;31m") \
	  LESS_TERMCAP_me=$(printf "\e[0m") \
	  LESS_TERMCAP_se=$(printf "\e[0m") \
	  LESS_TERMCAP_so=$(printf "\e[1;44;33m") \
	  LESS_TERMCAP_ue=$(printf "\e[0m") \
	  LESS_TERMCAP_us=$(printf "\e[1;32m") \
	  man "$@"
}


Better htop: gtop


https://github.com/aksakalli/gtop


Regular expression


Syntactic sugar for using regular expression with AWK.


function regex {
  gawk 'match($0,/'$1'/, ary) {print ary['${2:-'0'}']}'
}


Feedback sounds


Put any sound you like. It helps making human-computer interactions
  more human.


function yay {
  afplay $HOME/snd/yay.m4a > /dev/null 2>&1 &
}

function mwahaha {
  afplay $HOME/snd/mwahaha.mp3 > /dev/null 2>&1 &
}


Generate a gif from a screen record


I’ve always thought a fully fledged screen record is way too heavy
  when you just want to report basic interaction.


function gifgo {
  ffmpeg -i "$1" -pix_fmt rgb24 -r 10 -s 900x600 -f gif - | gifsicle --delay=15 --optimize=3 > "$1.gif"
}


Generate and display a QR code from the clipboard


A space is prepended before the first argument just for my own
  convenienve: if not, links are parsed as links by my phone and not as
  text (I prefer text).


function qrgo {
    tmp_file=$(mktemp).png
    qrencode  -o - "$(pbpaste)" > $tmp_file
    open $tmp_file
}


Grep json


brew install gron


Example of how to use it:


gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | fgrep "commit.author" | gron --ungron


It’s like jq but in less powerful but easier to use: gron’s primary
  purpose is to make it easy to find the path to a value in a deeply
  nested JSON blob when you don’t already know the structure; much of
  jq’s power is unlocked only once you know that structure.


Query json with jq


brew install jq


Here is the documentation.


Query csv or tsv with sql


brew install q


https://github.com/harelba/q


q "SELECT COUNT(*) FROM ./clicks_file.csv WHERE c3 > 32.3"

ps -ef | q -H "SELECT UID,COUNT(*) cnt FROM - GROUP BY UID ORDER BY cnt DESC LIMIT 3"


fzf, command-line fuzzy finder


https://github.com/junegunn/fzf


[ -f ~/.fzf.zsh ] && source ~/.fzf.zsh


    
  
  • C-t Paste the selected files and directories onto the command-line
    • 
  
  • C-r Paste the selected command from history onto the command-line (C-r again for chronological order)
    • 
  
  • M-c Move into the selected directory
    • 



Python


I may have installed Python but I’ve quite seldom used it. However the
  warning message looks like something which is awfully boring to debug
  so I put this warning here as it will help me in six months.


$ brew doctor

Warning: Putting non-prefixed findutils in your path can cause python builds to fail.



path+=('/usr/local/opt/python/libexec/bin'); export PATH


Haskell


path+=("$HOME/.cabal/bin"); export PATH


Go language


See what I put into my =HOME=.


export GOPATH="$HOME/go"


Lilypond


I put it in $HOME/.zshenv because it’s fair enough to consider
  lilypond as a command. As so, it should be widely available.


function lilypond {
  /Applications/LilyPond.app/Contents/Resources/bin/lilypond "$@"
}


Rust


path+=("$HOME/.cargo/bin"); export PATH
rustc --print sysroot


path+=("$HOME/.cargo/bin"); export PATH
export RUST_SRC_PATH=<<rust-source-path()>>/lib/rustlib/src/rust/src


Count line of codes in a directory


cargo install loc


Clojure


Clojure can be installed through brew. Leiningen settings (in
  $HOME/.lein) are in its own separate repository.


Custom Maven location


Maven location should not be custom. However I feel like it’s more
  explicit to set it here so you have no doubt on my intention (look,
  I’m honest, I’m setting it here ^^).


export M2_HOME=$HOME/.m2/repository # Default value, made explicit


Confluent platform


I’ve installed confluent-oss.


Default port configuration for localhost.



  
ComponentPort

  
Apache Kafka brokers (plain text)9092

  
Confluent Control Center9021

  
Kafka Connect REST API8083

  
REST Proxy8082

  
Schema Registry REST API8081

  
ZooKeeper2181




Install confluent platform with:


brew install confuent-platform



brew --prefix confluent-platform


export CONFLUENT_HOME=<<confluent-home()>>


According to the documentation I install the command line can be installed with:


curl -L https://cnfl.io/cli | sh -s -- -b $(brew --prefix)/bin



Start the platform with


confluent local start


After reading https://github.com/confluentinc/confluent-cli I feel it
  could be better to add confluent home in my env vars.


Android


According to getting started of React native:


export ANDROID_HOME=$HOME/Library/Android/sdk
export PATH=$PATH:$ANDROID_HOME/tools
export PATH=$PATH:$ANDROID_HOME/tools/bin
export PATH=$PATH:$ANDROID_HOME/platform-tools
expect-configured-environment-variable PIOTR_YUXUAN_GRADLE_SIGNING_KEY_PASSWORD


Some scripts also needs these data but run in bash.


export ANDROID_HOME=$HOME/Library/Android/sdk
export PATH=$PATH:$ANDROID_HOME/tools
export PATH=$PATH:$ANDROID_HOME/tools/bin
export PATH=$PATH:$ANDROID_HOME/platform-tools
expect-configured-environment-variable PIOTR_YUXUAN_GRADLE_SIGNING_KEY_PASSWORD


Reproductible configuration shows the version of Android Studio I run.


PostgreSQL


Command line tools are made available with:


path+=('/usr/local/opt/postgresql/bin'); export PATH


CLI tools


export PGDATA="/usr/local/var/postgres"


Socket error, can’t connect


If PostgreSQL looks started in brew services list but you can’t
  acquire a connection to its socket, perhaps it’s because you’ve had a
  shutdown which wasn’t clear for postgre. In this case, do the following:


rm /usr/local/var/postgresql/postmaster.pid
brew services restart postgresql



Cassandra, C*


https://gist.github.com/Micka33/89897e1490240a56c036


Properties: /usr/local/etc/cassandra
  Logs: /usr/local/var/log/cassandra
  Data: /usr/local/var/lib/cassandra/data


Schemata, schema registry


export SCHEMA_REGISTRY_URL=http://localhost:8081
export SCHEMA_REGISTRY_URI=http://localhost:8081


Copy or reset schemas topic in local


function destructive_reset_local_schemas_to_latest_production {
  confluent local destroy
  echo "\$(confluent local current | tail -n1)=$(confluent local current | tail -n1)"
  confluent local start
  echo 'Kafka topics after $(confluent local start):'
  kafka-topics \
  --zookeeper localhost:2181 \
  --list
  echo 'Creating _schemas-test'
  kafka-topics \
    --zookeeper localhost:2181 \
    --create \
    --topic _schemas-test \
    --partitions 1 \
    --replication-factor 1 \
    --config cleanup.policy=compact
  DUMP_FILE=$(gmktemp -d)/_schemas_$(date +%Y%m%d%H%M).log
  echo "DUMP_FILE=$DUMP_FILE"
  echo "Are you using the VPN?\n"
  echo "Consume production schema topic"
  kafka-console-consumer \
    --bootstrap-server bootstrap.server:9092 \
    --topic _schemas \
    --from-beginning \
    --property print.key=true \
    --timeout-ms 15000 \
    1> $DUMP_FILE
  echo "kafka.consumer.ConsumerTimeoutException is fine: it means the topic"
  echo "has been read to the end and no more message has been found, hence"
  echo "causing the timeout.\n"
  echo "Produce messages to local topic _schemas-test"
  cat $DUMP_FILE | kafka-console-producer --broker-list localhost:9092 --topic _schemas-test --property parse.key=true
  echo "Restart schema registry"
  confluent local stop schema-registry
  confluent local start schema-registry
}


Amazon environment variables


expect-configured-environment-variable AWS_ACCESS_KEY_ID
expect-configured-environment-variable AWS_SECRET_ACCESS_KEY


IntelliJ settings repository


Token for IntelliJ to manage its settings repository.


expect-configured-environment-variable INTELLI_J_SETTINGS_REPOSITORY


Reduce the size of my music library


Here is how to extract audio for a single file and


if ffmpeg -i a.mkv -vn -acodec copy a.ogg ; then
    rm a.mkv
else
    echo "Failed for a.mkv"
fi


The result is:



  
namesize

  
a.mkv7.8M

  
a.ogg2.0M




So I’m pretty convinced. Moreover, according to Wikipedia, « being a
  container format, Ogg can embed audio and video in various formats »
  so I guess it doesn’t lower quality too much. I’ve quickly checked
  with Audacity: audio rate is at 48k so it’s nice.


Let’s extract audio for all files. I want to keep things simple and
  use two functions: one to search for files, and another one to convert
  them. However it’s hitting some limitations of find and sub shell
  spawning (which doesn’t import functions) so let’s do it in a dirty way.


 find . -type f -name "*.mkv" -exec bash -c 'FILE="$1";
   if ffmpeg -i "$FILE" -vn -acodec copy "$FILE.ogg" ; then
	rm "$FILE"
   else
	echo "Failed for $FILE"
   fi' _ '{}' \;


Before I actually launch it on my music files, let’s try it. Pick a
  bunch of random files, copy them into a folder in /tmp and record
  how much they weight. dh -sh . returns 557M. Note that it includes
  also non-mkv files. The actual mkv files are 453M.


Now let’s run the little script. The ogg replacement are 145M. Quite
  good, we’ev got a 66% size reduction! The other files are
  untouched. OK, it passes the test, let’s release it.


The initial size of ~/Documents/snd/music is 179G, that’s big! If
  the same ratio of 66 is kept, I expect a minimal final size of 60G
  (just like all files would be mkv).


Oups, I’ve just thought I could have some actual videos in this music,
  for example a ballet music. Well, I guess it’ll be only a few of them,
  and music will remain. I don’t often change names, and YouTube video
  names often contain a code to uniquely identify the video. It should
  be enough to recover the few I could actually badly miss.


Now it’s around 98G after this first command has completed. We’ve
  already freed 81G for binaries or code! Now the free space ratio of my
  mac 1T drive is around 28%. I guess it’s enough for now. Next time
  I’ll try to compress other file formats.


OpenJDK in spite of Oracle JDK


No joke, bro.


export OPEN_JDK_HOME="$HOME/.jenv/bin"
path+=($OPEN_JDK_HOME); export PATH

eval "$(jenv init -)"