General Concepts

Conventions and standards

3-2-1 Data Backup Rule

• 3 - at least 3 copies (one primary, two copies)
• 2 - two different technologies (e.g. different filesystem, different software, or different media types)
• 1 - one copy should be off-site/offline

Semantic Versioning: aa.bb.cc

• aa - major release, quite different from previous version
• bb - minor release, add a new feature
• cc - patch release, a bug fix

The Twelve Factor App(source)

1. Codebase - One codebase tracked in revision control, many deploys
2. Dependencies - Explicitly declare and isolate dependencies
3. Config - Store config in the environment
4. Backing services - Treat backing services as attached resources
5. Build, release, run - Strictly separate build and run stages
6. Processes - Execute the app as one or more stateless processes
7. Port binding - Export services via port binding
8. Concurrency - Scale out via the process model
9. Disposability - Maximize robustness with fast startup and graceful shutdown
10. Dev/prod parity - Keep development, staging, and production as similar as possible
11. Logs - Treat logs as event streams
12. Admin processes - Run admin/management tasks as one-off processes

Environment variables

Linux:

• How To Read and Set Environmental and Shell Variables on Linux(Digital Ocean)

Windows:

• about_Environment_Variables
• HowTo: Set an Environment Variable in Windows - Command Line and Registry
• Set PATH and other environment variables in Windows 10

• Environment is an area that the shell builds every time that it starts a session that contains variables that define system properties.
• The environment provides a medium through which the shell process can get or set settings and, in turn, pass these on to its child processes. It implemented as strings that represent key-value pairs. If multiple values are passed, they are typically separated by colon(:) characters: KEY=value1:value2:.... If the value contains significant white-space, quotations(as double as single quotes) are used: KEY="value with spaces", KEY='value with spaces'.
• An environment variable is a storage location that has a name(key) and a value. The keys in these scenarios are variables. They can be one of two types, environmental variables or shell variables(see below).
• Benefit to things that should keep secret or need to be dynamic, like API keys, PORT, database url.

Common Environment Variables

Variable	Description
EDITOR	The program to run to perform edits.
HOME	The Home directory of the user.
LOGNAME	The login name of the user.
MAIL	The location of the user's local inbox.
OLDPWD	The previous working directory.
PATH	A colon separated list of directories to search for commands.
PAGER	This program may be called to view a file.
PS1	The primary prompt string.
PWD	The present working directory.
USER	The username of the user.

Common commands

• echo $VARIABLE_NAME or printenv VARIABLE_NAME - examine specific variable
• env or printenv - examine all the environmental variables that are set

• set - list of all shell variables, environmental variables, local variables(and also shell functions in Bash)

  • (#!bash set -o posix; set) - for Bash: clean up the output by specifying that set should operate in POSIX mode, which won’t print the shell functions by executing this in a sub-shell(by wrapping the whole command with parenthesis) so that it does not change our current environment.

  • comm -23 <(set | sort) <(env | sort), comm -23 <(set -o posix; set | sort) <(env | sort)(in Bash) - list of only shell variables

    Warning

    This will likely still include a few environmental variables, due to the fact that the set outputs quoted values, while the printenv and env do not quote the values of strings.

• KEY=VALUE or export KEY=VALUE - set shell or environmental variable(see below)

• env VAR1="value1" VAR2="value2" command_to_run command_options - modify the environment that programs run in by passing a set of variable definitions into a command
• printenv | grep VARIABLE_NAME - check if variable is an environmental variable
• set | grep VARIABLE_NAME - check if variable is a shell variable
• export -n VARIABLE_NAME - demote an environmental variable, i.e. set to shell variable
• unset VARIABLE_NAME - remove or delete an environment variable

Types of environment variables

• Shell(Local?) Variables - defined using KEY=VALUE format. Only effects the current running process, e.g. within the running script or shell in which they were set or defined. There are some predefined by shell vars of such type and they are often used to keep track of ephemeral data, like the current working directory(PWD).

• Environmental(Global?) Variables - defined using export KEY=VALUE format. export exports the variable assignment to child processes of the shell in which the export command was ran. In general, when a process is started it inherits the exported environment variables of the process that spawned it.

• Local vs. Global variables example:

  $ echo $PAGER
                         # PAGER is not defined
  $ PAGER=less           # let's difine it
  $ echo $PAGER
  less                   # now PAGER defined locally
  $ bash                 # running a sub-shell
  $ echo $PAGER
                         # PAGER is not defined in child process
  $ exit                 # exit from a sub-shell to parent shell
  exit
  $ export PAGER=less    # let's define PAGER globally
  $ bash                 # running a sub-shell
  $ echo $PAGER
  less                   # now PAGER defined in child process too!
  $ exit
  exit
  $
  

Tip

If you need envs(wether they are local or environmental) that are defined in some file to be available in current running interactive or non-interacrive(e.x. when running script) shell you should use .(source - optional for Bash) command, e.g.: . ./file-with-envs.sh(this actually executes the script commands in the current shell environment)

Levels of environment variables

Definitions:

• Login Shell - is a shell session that begins by authenticating the user, e.g. if you signing into a terminal session or through SSH and authenticate.
• Non-Login Shell -is a new shell session from within your authenticated session.
• Interactive Shell - is a shell session that is attached to a terminal.
• Non-Interactive Shell - is a shell session that is not attached to a terminal session. It’s most often run from a script or similar. It is important to note that this often influences your PATH variable.

• Detect the type of shell:

  ## Detect login/non-login shells:
  shopt -q login_shell && echo 'login' || echo 'not-login'    # for Bash only
  # or
  shopt | grep login_shell                                    # for Bash only
  # or
  echo $0                                                     # if output prepended with '-' then it is a login shell, e.g. '-zsh'
  
  ## Detect interactive/non-interactive shells:
  [[ $- == *i* ]] && echo 'Interactive' || echo 'not-interactive'
  

Reference:

• Unix Shells: Bash, Fish, Ksh, Tcsh, Zsh startup files
• Difference between Login Shell and Non-Login Shell?(Unix & Linux Stack Exchange)
• Zsh/Bash startup files loading order (.bashrc, .zshrc etc.)

• Zsh - Startup/Shutdown files(ArchWiki)

• Shell Startup Graph

  

  https://blog.flowblok.id.au/2013-02/shell-startup-scripts.html

1. System-wide - available for any user, session, app etc. Set in folowing files:

   • /etc/environment - This file is parsed by pam_env module. Syntax: simple "KEY=VAL" pairs on separate lines.

   • for Login Shells:

     • Bash(read by shell in this order! ^source): /etc/profile -> logged-in user dotfiles

     • Zsh(read by shell in this order!): /etc/zshenv -> logged-in user dotfile -> /etc/zprofile -> logged-in user dotfile -> /etc/zshrc -> logged-in user dotfile -> /etc/zlogin -> logged-in user dotfile

   • for Non-Login Interactive Shells:

     • Bash(read by shell in this order!): /etc/bash.bashrc -> logged-in user dotfiles

     • Zsh(read by shell in this order!): /etc/zshenv -> logged-in user dotfile -> /etc/zshrc -> logged-in user dotfile

   • for Non-Login Non-Interactive Shells(scripts etc.):

     • Bash: BASH_ENV environmental variable

     • Zsh(read by shell in this order!): /etc/zshenv -> logged-in user dotfile

   Note

   Restart system after modifying these system files so changes will take effect.

2. User - available for current logged-in user. Set in following dotfiles of the shell that is currently used by that user in a KEY=VALUE format:

   Note

   IMHO: export KEY=VALUE can be used too, but thus it exports the variable assignment to child processes of the shell which is in my opinion quite unneseccairy for most of defined envs because each time we run new sub-shell those envs get sourced by the shell.

   • for Login Shells:

     • Bash(read by shell in this order but it executes only the first of those files found! ^source): system-wide settings files -> ~/.bash_profile -> ~/.bash_login -> ~/.profile

       Tip

       Put these in ~/.profile at the top in order to source ~/.bashrc:

       # ~/.profile must include ~/.bashrc, but only if the shell is interactive and is bash but not if the login shell is some other shell
       case "$-" in *i*)
           if
               test "$BASH_VERSION" &&\
               test "${0#-}" != sh &&\
               test -r "$HOME"/.bashrc
           then
               . "$HOME"/.bashrc
           fi
           ;;
       esac
       

       • Configuring your login sessions(for 5 different environments!) with dot files
       • Difference between .bashrc and .bash_profile(SuperUser)

     • Zsh(read by shell in this order!): system-wide settings file -> $ZDOTDIR/.zshenv -> system-wide settings file -> $ZDOTDIR/.zprofile -> system-wide settings file -> $ZDOTDIR/.zshrc -> system-wide settings file -> $ZDOTDIR/.zlogin

   • for Non-Login Interactive Shells:

     • Bash(read by shell in this order!): system-wide settings files -> ~/.bashrc

     • Zsh(read by shell in this order!): system-wide settings file -> $ZDOTDIR/.zshenv -> system-wide settings file -> $ZDOTDIR/.zshrc

   • for Non-Login Non-Interactive Shells(scripts etc.):

     • Bash: BASH_ENV environmental variable

     • Zsh(read by shell in this order!): system-wide settings -> $ZDOTDIR/.zshenv

   Note

   If $ZDOTDIR is not set, $HOME is used instead.

   Tip

   When you have both Bash and Zsh shells in your system it is convenient to define $PATH(and other system envs) and also some other user-defined envs in ~/.profile and source it through ~/.zprofile by adding [[ -e ~/.profile ]] && emulate sh -c '. ~/.profile' line at the top of the file(Zsh not hitting ~/.profile(SuperUser thread comment)).

   There is also other complicated scenario with multiple shells and ways to arrange your envs in one place:

   • Xsh - A simple framework for shell configuration management.
   • Is there a ".bashrc" equivalent file read by all shells?(SuperUser thread comment)
   • Shell startup scripts

3. Shell Session - available for current shell session and its child processes. Set by using environmental variables.

4. Application/Script - available for current running app/script process only. Set by using shell variables, e.g.:

   # Prepending Variables - you can prepend ANY number of variables before running a command.
   
   PORT=3000 node server.js    # set port of server
   

   # myscript.sh
   
   MY_VAR="Hello World!"
   echo MY_VAR    # Hello World!
   

Regex

• RegExp(4pda)
• Regex in Java Script(MDN)
• Regular Expression HOWTO(python.org)
• Regex tutorial — A quick cheatsheet by examples(Medium)
• Regex cookbook — Top 10 Most wanted regex(Medium)
• RegexOne Tutorial

• Шпаргалка по регулярным выражениям

  

---

Last update: 2023-01-26