-
Abstract: This guide provides best practices for writing clear, maintainable, and robust Zsh scripts. Aimed at helping developers transition from Bash to Zsh, particularly on macOS, this guide offers practical advice and examples for creating standardized and efficient scripts.
-
Copyright This text of this guide is Copyright ©️2024 by Christopher Allen, and is shared under spdx:CC-BY-SA-4.0 open-source license. All the example code is relenquished to the public domain under spx:CC0-1.0.
-
Tags: #zsh #scripting #cli #opinionated #guide #bestpractices #tips
-
Version: 0.1.2 (2024-05-05) - Minor Update (for information on this versioning scheme, see Status & Versioning)
If you like these tools, my writing, my advocacy, my point-of-view, I invite you to sponsor me.
It's a way to plug into an advocacy network that's not focused on the "big guys". I work to represent smaller developers in a vendor-neutral, platform-neutral way, helping us all to work together.
You can become a monthly patron on my GitHub Sponsor Page for as little as $5 a month.
But please don’t think of this as a transaction. It’s an opportunity to advance the open web, digital civil liberties, and human rights together. You get to plug into my various projects, and hopefully will find a way to actively contribute to the digital commons yourself. Let’s collaborate!
-- Christopher Allen <ChristopherA@LifeWithAlacrity.com> Github: @ChristopherA X/Twitter: @ChristopherA
- Introduction
- Organizing a Zsh-based Project Repository
- Strategic Use of Case in Names
- Best Practices for Zsh Script File Names
- Best Practices for Zsh Variable and Function Names
- Avoiding Overuse of Global Variables
- Some Useful Environment Variables and Conventions
- Operating System Environment Variables
- System and Shell Environment Variables
- macOS Environment Variables
- User Environment Information
- Shell Environment Variables
- Zsh Specific Environment Variables
- Working Directory Information
- Script Environment
- Caller Script Information
- Process Information
- Script Arguments
- Git Environment Variables (local and global)
- Other Common Environment Variables
- Snippets of Code with Naming Conventions Applied
- macOS and Zsh
- To Be Continued
This is my opinionated guide to Zsh best practices.
To be clear, I've never produced a complicated Zsh script in a major production environment, so I can't quite call myself an expert, but here is what I've learned through my experience and research.
One of the major reasons that I've had to create this guide is that macOS transitioned away from bash
to default to zsh
with the release of macOS Catalina 10.15 in 2019, and their choice to not keep up with newer versions of bash
in their default environment. This broke some of my favorite scripts, utilities, tools, and I decided it was time to just fully switch.
My goal in all of these recommendations is to help create clear, maintainable, and robust Zsh scripts.
My approach initially focuses on naming conventions, file organization, and coding standards to enhance readability and maintainability. This guide is meant to provide a consistent framework for Zsh scripting, but it's not set in stone. I welcome feedback and am open to learning from others. If you disagree with any of these practices, I'd love to understand your reasoning and am open to changing my mind.
I've drawn inspiration from other opinionated guides across various programming languages and frameworks which emphasize consistency and clarity in coding practices. By adopting similar principles, we can create a more standardized approach to Zsh scripting that benefits everyone in the long run.
Let's dive in and explore these best practices together, keeping in mind that the ultimate goal is to write better, more maintainable code.
For a project involving multiple scripts, consider the following structure:
(Note: details of the inside organization of ./repo/
folder is a work-in-progress)
my-project/
├── README.md
├── .gitignore
├── Makefile
├── bin/
│ ├── build-project.sh
│ ├── deploy-project.sh
│ └── test-project.sh
├── config/
│ ├── setup-environment.zsh
│ └── configure-database.zsh
├── lib/
│ ├── utility-functions.zsh
│ └── string-manipulations.zsh
├── repo/
│ ├── config/
│ │ ├── allowed_commit_signers.txt
│ │ ├── allowed_tag_signers.txt
│ │ └── trust-manifest.envelope
│ ├── hooks/
│ │ ├── pre-commit.sh
│ │ ├── pre-push.sh
│ │ ├── commit-msg.sh
│ │ ├── post-merge.sh
│ │ └── (etc.)
│ ├── scripts/
│ │ ├── verify_commit_signers.sh
│ │ ├── verify_tag_signers.sh
│ │ ├── verify_release_signers.sh
│ │ ├── deploy_helpers.sh
│ │ ├── setup_environment.sh
│ │ └── security_checks.sh
│ ├── pipeline/
│ │ ├── github_actions.yml
│ │ ├── gitlab_ci.yml
│ │ └── bitbucket_pipelines.yml
├── tests/
│ ├── test-build-project.sh
│ ├── test-deploy-project.sh
│ └── function_deploy-TEST.sh
├── docs/
│ ├── CONTRIBUTING.md
│ ├── CHANGELOG.md
│ └── LICENSE
- README.md: Provides an overview and key information about the project. It should be located in the root directory to be recognized by GitHub.
- .gitignore: Specifies intentionally untracked files to ignore.
- Makefile: Automates the build process with
make
commands. - bin/: Contains executable scripts for building, deploying, and testing the project.
- config/: Contains configuration scripts and settings.
- lib/: Contains library scripts with reusable functions.
- repo/:
- config/: Contains configuration files for signing and trust management.
- hooks/: Contains Git hook scripts for various Git lifecycle events.
- **scripts
/**: Contains utility scripts for verification and setup tasks.
- pipeline/: Contains CI/CD pipeline configuration files for different platforms (GitHub Actions, GitLab CI, Bitbucket Pipelines).
- tests/: Contains test scripts for various project components and functions.
- docs/: Documentation files.
- CONTRIBUTING.md: Guidelines for contributing to the project.
- CHANGELOG.md: A log of changes for each version of the project.
- LICENSE: The project's license file.
Specific choices of case in names can significantly enhance efficiency and readability when writing and using Zsh code using the command line interface. Here’s my approach:
-
Lower Case Preference: One reason for choosing to use lower case filenames for scripts is that most command names in Unix-like systems are also in lower case (e.g.,
ls
,cd
,grep
), and if scripts are going to be called like these commands, they too should be lower case. Another reason is that usingMixedCamelCase
in directory and file names can slow down tab completion in the shell. Thus, I strongly prefer usinglowercase
filenames for that efficiency. Lower case is also somewhat advantageous for auto-completion in many code editors, however, most tend to be more tolerant of different case, so I can make different choices for variables and functions. -
Use Upper Case Strategically: Slowing down tab completion can be useful to avoid mistakes. For example, using
connectAlice.sh
andconnectBob.sh
requires intentional effort to execute the correct file (capital-A vs capital-B respectively), thus I am less likely to accidentally execute the wrong script. -
Use
lower_snake_case
for Readability: Being able to quickly scan a significant amount of code requires some structure to enhance readability. SinceCamelCase
isn't an option for me (in particular for filenames), I largely preferlower_snake_case
for readability. This format ensures that filenames are clear and easy to understand. -
Avoid
lower-kebab-case
for Variables and Functions: Zsh variable and function names can't contain dashes or periods as they aren't valid identifiers. Use underscores insmall_snake_case
orCamelCase
instead.lower-kebab-case
can still be used for filenames as it helps in distinguishing parts of the name, but not for code. -
Combine Separators Strategically: Mixing name separators (
_
or-
) can be useful for complex names. For instance,project_charles-build_production-part_one
allows selective double-clicking to highlight specific parts of the name (for copy or to paste over), i.e.project_charles
,build_production
, andpart_one
. -
Consistency Over Style: The exact naming style (whether
CamelCase
,snake_case
, or another convention) is less important than being consistent throughout your codebase. This consistency helps in maintaining readability and understanding, especially in collaborative environments.
- Versioned Scripts:
process_data-v1.sh
andprocess_data-v2.sh
- Test Scripts:
get_database-TEST.sh
- Configurations:
config_server-main.sh
andconfig_server-backup.sh
When naming Zsh scripts, consider the following:
-
Use Lower Case: For consistency with other shell commands, use lower case names, but sometimes it can be strategically useful to use mixed case.
-
Use the Appropriate Extension:
- Use
.sh
for command-line executable scripts. This relies on the#!/usr/bin/env zsh
shebang to specify the Zsh interpreter shell. - Use
.zsh
for libraries, function tests, and other scripts that are intended to be sourced or included in other scripts.
Example:
deploy_project.sh
for an executable script.string_manipulations_functions.zsh
for a library of functions.
- Use
-
Use Descriptive Names: The script's name should clearly describe its purpose or functionality. This helps users understand what the script does without having to open it.
Example:
setup_database.sh
for a script that sets up the database.backup_database.sh
for a script that handles database backups.
-
Prefix with Action Words: Try to start the script name with a verb that indicates the action performed by the script, followed by an object. This makes it clear what the script is supposed to do.
Example:
install_packages.sh
clearly indicates that the script installs packages.remove_temp_files.sh
clearly indicates that the script removes temporary files.
-
Keep It Short but Clear: While the name should be descriptive, it should also be concise. Aim for a balance between clarity and brevity.
Example:
sync-backup.sh
instead ofsynchronize-backup-directory.sh
.
-
Avoid Overuse of Acronyms & Short Terms: While acronyms can shorten names, overusing them can make script names cryptic. Use them judiciously and ensure they are commonly understood.
Example:
setup-command-line-envelope.sh
is clearer thansetup-cl-env.sh
(especially as -env often refers to the environment).
-
Avoid Spaces and Special Characters: Use only alphanumeric characters, underlines, and hyphens. Avoid spaces and non-ASCII characters (in particular those found in macOS), as they can cause issues in some environments and platforms, and in general make the script harder to use.
Example:
generate_report.sh
instead ofgenerate report.sh
.
-
Test Script Naming:
- Use
<script_name>-TEST.sh
for functional tests of the script<script_name>.sh
. - Use
<function_name>-FTEST.sh
for tests of a specific function() in a script.
Example:
deploy_project-TEST.sh
for testing thedeploy_project.sh
script.parse_table-FTEST.sh
for testing theparse_table()
function.
- Use
-
Prefix for Related Scripts:
- If there are a series of related scripts as part of a project or system, use a prefix to identify and connect them together.
Example:
- zutil_script_template.sh
for the Zsh Utilities script template.
- zutil_git_tool.sh
for the Zsh Utilities git tool.
- zutil_script_template-TEST.sh
for the functional tests of the zutil_script_template.sh
.
- zutil_parse_params-FTEST.sh
for the script that tests the zutil_parse_params()
function.
- Prefix for Special Uses:
- Prefix a file with an underscore (
_
) if it is intended not to be executed directly but instead called or sourced by other scripts.
- Prefix a file with an underscore (
Example:
- _config.zsh
for configuration settings sourced by other scripts.
- _zutil_common_utilities.zsh
for the common utility functions used by zutil_
project scripts.
- Versioning: If you have multiple versions of a script, include the version number in the file name for any alternative versions. I prefix these with a hyphen
-
for strategic double-click selection.
Example:
- deploy_website.sh
for the working release.
- deploy_website-legacy
for the legacy deploy script.
- deploy_website-v1.2.sh
for version 1.2 of the deploy script.
These are requirements of Zsh:
-
Avoid Hyphens and Periods in Variable and Function Names: In Zsh, variable and function names cannot contain dashes or periods as they are not valid identifiers. Use underscores or CamelCase instead.
-
Avoid Special Characters in Variable and Function Names: Avoid characters like
!
,@
,#
,$
,%
,^
,&
,*
,(
,)
,+
,=
,{
,}
,[
,]
,|
,\
,:
,;
,"
,'
,<
,>
,?
,/
as they have special meanings in Zsh and other Unix-like shells. Using them in variable or function names can cause unpredictable behavior or errors.
These remaining formatting choices are my personal preferences, developed with a primary focus on general readability to ensure the code is easy to understand at a glance. Additionally, these conventions aim to reduce errors by allowing quick differentiation between functions and the various types of variables. Finally, these guidelines emphasize consistency throughout the codebase, making it easier to maintain and collaborate on projects.
-
Global Variables: Use all uppercase with underscores (UPPER_SNAKE_CASE), e.g.
$VERBOSE_MODE
. These are often exported for use by other processes or scripts. -
Scoped Variables: Use Mixed_Snake_Case for variables that will persist in the current execution environment, such as sourced scripts and called functions, e.g.
$Scoped_Variable
. Ensure that scoped variable names are at least two words for clarity. -
Local Variables: Use CamelCase for local variables defined within specific functions or small blocks of code, e.g.
$LocalScopedVariable
. For short, single-word local variables, using lowercase is acceptable, especially if the variable type is clear from context, such astypeset -i count
. -
Function Names: Use
lowerfirst_Snake_Case
for function names to align with script names and make them distinct from variables, e.g.log_Message
. -
Private/Internal Variables & Functions: Prefix with an underscore (
_
) to indicate the variable or function is private to a series of interrelated scripts (e.g. the global$_ZUTIL_DBUG
, or internal$_debug
inside a local function is different than locally scoped$Debug
).# Public function example log_Message() { local message=$1 echo "$(date): $message" >> "$LOG_FILE_FULL_PATH" # Private function example _log_Internal() { local _internalMessage=$1 echo "Internal: $_internalMessage" } # Calling the private function _log_Internal "This is a test internal message" }
In Zsh, typeset
, in combination with appropriate setopt
options, is a versatile command for declaring and managing variables. It provides better control over variable scoping as compared to global
and local
. It can also be combined with various options to achieve different functionalities to ensure that variables have the desired properties, such as being local, read-only, global, integers, arrays, or associative arrays.
-
Declare a Variable Scoped to the Current Script Context: Using
typeset
explicitly defines a variable within the current script or function context, making its scope clear and avoiding ambiguity.These are equivalent, but don't be vague, use
typeset
:# bad example my_Function() { ScriptVar="value" echo "Script variable: $ScriptVar" }
Do be explicit:
# good example my_Function() { typeset ScriptScopedVar="value" echo "Script scoped variable: $ScriptScopedVar" }
-
Declare Global Variable: The
-g
option withtypeset
makes the variable global, allowing it to be accessed and modified from anywhere in the script, including within functions.typeset -g GLOBAL_VAR="global_value" echo "Global variable: $GLOBAL_VAR"
If the Zsh-specific
setopt warn_create_global
is set, Zsh will issue a warning. See details in Setting Useful Options withsetopt
. -
Declare Local Variable: Declaring a variable as local within a function ensures it is only accessible within that function, preventing unintended interactions with variables outside the function.
my_Function() { local localVar="value" echo "Local variable: $localVar" }
-
Declare Read-Only Variable: Using the
-r
option withtypeset
makes the variable read-only, meaning its value cannot be changed after it is set. This is useful for constants, ensuring that their values remain consistent throughout the script. Additionally, declaring variables as read-only reduces errors by preventing accidental modifications and enhances safety by protecting critical values from being altered. If you try to modify a read-only variable, Zsh will report an error indicating that the variable is read-only, and this error message will be sent to standard error (stderr).typeset -r ReadOnlyVar="constant" echo "Read-only variable: $ReadOnlyVar" # Attempt to modify the read-only variable ReadOnlyVar="new_value" # This will cause an error
Example error message when attempting to modify a read-only variable:
ReadOnlyVar: read-only variable: ReadOnlyVar
-
Declare Integer Variable: Using the
-i
option withtypeset
declares an integer variable, ensuring that only integer values can be assigned to it. This helps maintain the integrity of numerical data and prevents unexpected behavior due to invalid assignments. If you try to assign a non-integer value to an integer variable, Zsh will automatically convert it to an integer, typically resulting in0
. Note that this conversion does not generate an error message and is sent to standard output (stdout).typeset -i IntCounter=42 echo "Integer counter: $IntCounter" # Attempt to assign a non-integer value IntCounter="text" # This will convert "text" to 0 echo "Updated integer counter: $IntCounter"
Example behavior when assigning a non-integer value to an integer variable:
Updated integer counter: 0
-
Declare Array Variable: The
-a
option withtypeset
declares an array variable, allowing multiple values to be stored and accessed using index positions. Ensure proper initialization syntax to avoid issues. Note that accessing an out-of-bounds index will not generate an error but will return an empty value.typeset -a ArrayVar=("element1" "element2") echo "Array variable: ${ArrayVar[@]}" # This will return an empty value, not an error echo "Accessing out-of-bounds index: ${ArrayVar[10]}"
-
Declare Associative Array Variable: Using the
-A
option withtypeset
declares an associative array variable, allowing key-value pairs to be stored and accessed. Ensure unique keys and proper quotation to avoid syntax errors and unintended behavior. Accessing a non-existent key will not generate an error but will return an empty value. If there is a syntax error, it will be sent to standard error (stderr).typeset -A AssocArray AssocArray[key1]="value1" AssocArray[key2]="value2" echo "Associative array: ${AssocArray[key1]}, ${AssocArray[key2]}" # Accessing a non-existent key echo "Non-existent key: ${AssocArray[nonExistentKey]}"
Example behavior when accessing a non-existent key:
Non-existent key:
You can combine multiple typeset
options to achieve various effects. For example:
-
Declare a Read-Only Global Variable:
typeset -gr GLOBAL_READ_ONLY_CONSTANT="constant_value" echo "Global constant: $GLOBAL_READ_ONLY_CONSTANT"
-
Declare a Local Integer Variable:
my_Function() { typeset -i localInt=100 echo "Local integer: $localInt" }
You can use typeset
to handle and parse an array of parameters passed to a function efficiently. Here's an example of how to parse parameters in a function:
Suppose you have a function that processes user information, where each user is represented by a set of attributes:
process_Users() {
typeset -a Users_Array=("$@") # Initialize an array with all passed parameters
typeset -i idx=1
while (( idx <= $#Users_Array )); do
typeset Name="${Users_Array[idx]}"
typeset Age="${Users_Array[idx+1]}"
typeset Email="${Users_Array[idx+2]}"
echo "Processing user:"
echo "Name: $Name"
echo "Age: $Age"
echo "Email: $Email"
(( idx += 3 )) # Move to the next set of user attributes
done
}
# Call the function with user data
process_Users "Alice" 30 "alice@example.com" "Bob" 25 "bob@example.com"
Associative arrays are very useful for handling key-value pairs, such as configuration parameters or settings. Here is an example of how to use an associative array to manage and parse configuration parameters in a function.
Suppose you have a function that processes configuration settings for an application:
process_Config() {
# Initializes an associative array `Config` with the parameters passed to the function.
typeset -A Config_Associative_Array=("$@")
local key
echo "Processing configuration settings:"
# Iterate over the keys of the associative array and prints each key-value pair.
for key in "${(@k)Config_Associative_Array}"; do
echo "$key: ${Config_Associative_Array[$key]}"
done
}
# Call the function with configuration settings and pass configuration parameters as key-value pairs
process_Config "database_host" "localhost" "database_port" "5432" "username" "admin" "password" "secret"
To take full advantage of typeset
, you can set several useful options in Zsh:
-
setopt typeset_silent
: Preventstypeset
from printing variables in a global context. -
setopt local_traps
: Ensures traps set within a function are local to that function. -
setopt warn_create_global
: A warning is sent to standard error (stderr) if a global variable is created implicitly without thetypeset -g
option, helping to prevent accidental global variable creation and enhancing safety.setopt warn_create_global # Enable warning for implicit global variable creation # Example of implicit global variable creation that triggers a warning my_Function() { GLOBAL_VAR="global_value" # This will
trigger a warning echo "Global variable: $GLOBAL_VAR" }
my_Function
```
Example warning message sent to standard error (stderr) when implicitly creating a global variable:
```stderr
GLOBAL_VAR: created global parameter in function
```
-
setopt local_options
: Ensures thatnounset
,errexit
and other options set within a function, such asnoglob
,noclobber
, etc., are automatically restored to their previous values when the function exits, preventing unintended side effects on the global environment. Example: ```sh # Enable strict mode for the script set -o errexit -o nounset -o pipefailstrict_Function() { setopt local_options setopt errexit nounset echo "Inside function: strict mode enabled" # Uncommenting the next line will cause an error due to nounset # echo "Undefined variable: $undefined_var" } echo "Before function: strict mode enabled" strict_Function echo "After function: strict mode restored" ```
Global variables in shell scripting, including both Bash and Zsh, can become a crutch and lead to significant headaches. They often create unintended side effects and make the code harder to understand and maintain.
Here are some strategies and best practices to avoid the pitfalls of overuse of global variables to create more robust, maintainable scripts:
-
Use Local Variables: Whenever possible, declare variables within the
local
scope of functions. This confines their accessibility and modifications to the function itself and its child functions, thereby reducing the risk of unintended interactions with other parts of the script. -
Dynamic Scoping in Zsh: Zsh uses dynamic scoping, meaning that variables are visible within the function and any functions it calls. Use
local
ortypeset
to ensure variables are not inadvertently modified in other scopes. For example:my_Function() { local LocalVar="value" echo "Local variable: $LocalVar" }
-
Unique Naming Conventions: If you must use global variables, work hard to make them unique to avoid collisions. Use very specific names, such as
GIT_UTILITY_SCRIPT_DIR
, or add a private prefix, e.g.,_ZUTIL_DEBUG
, or both_ZUTIL_TOP_SCRIPT_DIR
. -
Set
warn_create_global
Option: Enable thewarn_create_global
option in Zsh to spot variables that are implicitly made global. This can help catch unintentional global variables:setopt warn_create_global
-
Limit Scope with Functions: Encapsulate your code in functions and minimize the use of variables outside these functions. This practice not only reduces the need for global variables but also enhances readability and reusability.
-
Use Environment Variables for Truly Global Needs: Only when a variable needs to be truly global across multiple scripts, should use use a global environment variable. However, do this sparingly and with clear documentation, as it can affect the global state of the user's environment:
export MY_GLOBAL_VAR="some_value"
-
Read-Only Variables: If a global variable is necessary and should not be modified, declare it as read-only to prevent accidental changes:
typeset -r READ_ONLY_GLOBAL="constant_value"
-
Documentation and Comments: Always document the purpose and scope of any global variables. If a function uses a global variable, make sure to document it in the function comments. This helps other developers (and future you) understand why the global variable exists and how it should be used.
These variables provide information about the operating system on which the Zsh shell is running. These variables are useful to help adjust the OS environment and provide information about the operating system's state and configuration. They are especially useful for scripts that need to perform operations based on the specific OS details.
$(uname -s)
: Contains the operating system type (e.g.,Linux
,Darwin
).$(uname -r)
: Provides the kernel version of the operating system (e.g.23.5.0
)$(uname -m)
: Specifies the hardware architecture of the machine (e.g.,x86_64
,arm64
).$(uname -v)
: Contains the release name of the operating system version. On my macDarwin Kernel Version 23.5.0: Wed May 1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000
# Read-only operating system type
typeset -r OsType=$(uname -s)
# Read-only kernel version
typeset -r OsKernelVersion=$(uname -r)
# Read-only machine hardware name
typeset -r OsMachineHardware=$(uname -m)
# Read-only operating system release name
typeset -r OsReleaseName=$(uname -v)
There are a number of environment variables about the specific computer system that are established before the script is executed. I prefix most of these with SYS_
, Sys_
, or sys_
for clarity.
$(uname -n)
: Provides the network node hostname of the machine, as returned byuname
command. NOTE: I don't use$(hostname)
as returned by the dedicatedhostname
command. Both should be the same, butuname
should be more consistent across different Unix-like systems.$PATH
: is a system-level environment variable as it is available to all users and processes, however, it is also a user environment variable that likely is different for each user.$fpath
: is a Zsh-specific shell environment variable, used by Zsh to locate function files.
Though ideally, these should not change during the execution of a script, for safety I copy them into a read-only variable scoped to the script context, thus I use Mixed_Snake_Case for their names.
I sometimes use the suffix _Init
when I initialize them if I believe that they might change (or my script might change) these values during runtime.
# Read-only network node hostname
typeset -r SysNodeHostname=$(uname -n)
# Initial read-only system PATH variable
typeset -r SysPathInit=$PATH
# Initial read-only Zsh function search path variable
typeset -r SysFpathInit=$fpath
If OsIsOsx
then I find setting these environment variables useful. I prefix these with Osx
.
# Read-only flag indicating if the operating system is macOS
typeset -r OsIsOsx=$( [[ $(uname -s) == "Darwin" ]] && echo true || echo false )
# Mac Device Hardware Model if the system is macOS
[[ $OsIsOsx == true ]] && typeset -r OsxHardwareModel=$(sysctl -n hw.model)
# Get Device Model Name if the system is macOS
[[ $OsIsOsx == true ]] && typeset -r OsxDeviceModelName=$(system_profiler SPHardwareDataType | awk -F ': ' '/Model Name/ { print $2 }')
# Get Device CPU if the system is macOS
[[ $OsIsOsx == true ]] && typeset -r OsxDeviceCPU=$(/usr/sbin/sysctl -n machdep.cpu.brand_string | sed s/"Apple "//)
# Set read-only environment variables for macOS version information if the system is macOS
[[ $OsIsOsx == true ]] && read OsxProductVersion OsxVersMajor OsxVersMinor OsxVersPatch <<<$(sw_vers -productVersion | awk -F. '{print $0 " " $1 " " $2 " " $3}') && typeset -r OsxProductVersion OsxVersMajor OsxVersMinor OsxVersPatch
[[ $OsIsOsx == true ]] && typeset -r OsxBuildVersion="$(sw_vers -buildVersion)"
These variables contain information about the current system user, thus are not prefixed with sys_
, so I use the prefix User_
.
$USER
: is the current user's name.$LOGNAME
: is the login name of the user. Note: The$USER
and$LOGNAME
variables typically contain the same value, but there can be differences.$USER
is often set by the shell and can change within a session, reflecting the effective user ID.$LOGNAME
, on the other hand, is usually set by the login process and reflects the original login name, remaining consistent even if the user switches to another account using commands likesu
.$HOME
: is the current user's home directory.$SHELL
: is the user's default shell, ensuring any shell-specific operations within the script use the correct shell.$TERM
: is the current terminal type, which is useful for scripts that handle terminal-specific functionality or output formatting. In non-interactive sessions,$TERM
might be unset or set todumb
, indicating no interactive terminal is available. (If$TERM
is blank, for safety, I set it todumb
.)$LANG
: is the user's language and locale settings, ensuring scripts that handle localization or internationalization are consistent with the user's environment. On macOS, it usually defaults toen_US.UTF-8
.$(id -u)
: is the user's unique identifier, useful for scripts that need to verify or log user actions.$(id -g)
: is the user's group identifier, important for scripts managing file permissions or group-specific actions.$(id -gn)
: is the user's primary group name.$(id -Gn | tr ' ' ',')
: captures all the group names the user belongs to, which can be useful for managing permissions or checking group memberships.$MAIL
: is the user's mail directory, included for completeness, which may be relevant for scripts that handle or check legacy Unix-style user email.
When I use any of these environment variables, for safety, I copy them into read-only variables first.
# Read-only current user's name
typeset -r UserName=$USER
# Read-only login name of the user
typeset -r UserLogname=$LOGNAME
# Read-only current user's home directory, normalized using realpath
typeset -r UserHome=$(realpath "$HOME")
# Read-only user's default shell
typeset -r UserShell=$SHELL
# Read-only user's terminal type (set to default `dumb` if non-interactive or nil)
typeset -r UserTerm=${TERM:-dumb}
# Read-only user's language/locale setting (set it to a default if $LANG if nil)
typeset -r UserLang=${LANG:-en_US.UTF-8}
# Read-only user's unique identifier
typeset -r UserUid=$(id -u)
# Read-only user's group identifier
typeset -r UserGid=$(id -g)
# Read-only user's primary group name
typeset -r UserGroup=$(id -gn)
# Read-only user's primary group names (comma-separated)
typeset -r UserGroups=$(id -Gn | tr ' ' ',')
# Read-only user's mail directory
typeset -r UserMail=$MAIL
These variables are common to most Unix-like shells, including Zsh. They help manage the shell environment and provide information about the shell session's state and configuration. These variables are useful for scripts that need to adapt their behavior based on the shell environment.
interactive
: Indicates whether the shell is interactive (true
orfalse
).login
: Indicates whether the shell is a login shell (true
orfalse
).monitor
: Indicates whether the shell is running with job control enabled (true
orfalse
).restricted
: Indicates whether the shell is running in restricted mode
(true
or false
).
# Current working directory path, and works with paths with spaces (macOS)
typeset -r ShellCurDir=$(realpath "$PWD")
# Indicates whether the shell is interactive
typeset -r ShellInteractive=$([[ -o interactive ]] && echo 'true' || echo 'false')
# Indicates whether the shell is a login shell
typeset -r ShellLoginShell=$([[ -o login ]] && echo 'true' || echo 'false')
# Indicates whether the shell is running with job control enabled
typeset -r ShellJobControl=$([[ -o monitor ]] && echo 'true' || echo 'false')
# Indicates whether the shell is running with restricted mode enabled
typeset -r ShellRestricted=$([[ -o restricted ]] && echo 'true' || echo 'false')
These variables are related specifically to the Zsh shell environment. They provide metadata and configuration details about the current Zsh shell session.
$ZSH_VERSION
: Contains the version number of the Zsh shell.$HISTFILE
: Specifies the file where the command history is saved.$HISTSIZE
: Defines the number of commands to remember in the command history.$SESSION_ID
: Provides a unique identifier for the current Zsh session.$options
: An array containing all the current options set in the shell.$module_path
: Specifies the directories to search for Zsh modules.$fpath
: Specifies the directories to search for Zsh function files.
# Zsh version
typeset -r ZshVersion=$ZSH_VERSION
# Path to the history file
typeset -r ZshHistfile=$(realpath $HISTFILE)
# Size of the history file
typeset -r ZshHistsize=$HISTSIZE
# The current session ID
typeset -r ZshSessionId=$SESSION_ID
# Array of shell options
typeset -r ZshOptions=($options)
# Module search path
typeset -r ZshModulePath=$(realpath $module_path)
# Function search path
typeset -r ZshFpath=($fpath)
These environment variables relate to the current working directory when the script is executed. The user's working directory may differ from the script's directory. Also, by default, when a non-interactive shell session starts, it inherits the working directory from the parent process that started it.
These variables technically could just be another prefixed by User_
or Shell_
, however, I use them a lot, so use the prefix Workdir_
. As the working directory may change during script execution, I copy the working directory details to read-only variables, which allows the script to change directories and restore the initial working directory if needed.
Paths are normalized using realpath
to resolve symbolic links and remove redundant slashes, ensuring consistency. If realpath
is not available on your system, you may need to install it or use an alternative method to achieve similar functionality. On macOS, realpath
is typically available.
$PWD
: Holds the current working directory.
# Initial read-only absolute path of the current working directory
typeset -r WorkdirPathInit=$(realpath "$PWD")
# Initial read-only name of the current working directory
typeset -r WorkdirNameInit=$(basename "${(Q)$(realpath "$PWD")}")
# Working directory variables are is not read-only they can be changed during script execution
WorkDirPath=$WorkdirPathInit
WorkDirName=$WorkdirNameInit
# Read-only absolute path of the parent directory of the current working directory
typeset -r WorkdirParentDirInit=$(realpath "${(Q)${PWD:h}}")
# Read-only absolute path of a specific subdirectory in the current working directory
typeset -r WorkdirSubdir=$(realpath "$(pwd)/subdir")
# Read-only name of a specific subdirectory in the current working directory
typeset -r WorkdirSubdirName=subdir
These variables provide metadata about the execution environment of the script itself. These they can't change during script execution,
These can't not change during script execution, there is no need for the _Init
suffix. For safety I also initialize them as read-only to prevent further modification. I also use realpath
to normalize these in case of symbolic links.
# Read-only absolute path of the current script's directory
typeset -r ScriptDir=$(realpath ${0:A:h})
# Read-only name of the current script without the path
typeset -r ScriptName=${${0##*/}}
# Read-only absolute path of the current script
typeset -r ScriptPath=$(realpath ${0:A})
# Read-only absolute path of the parent directory of the current script's directory
typeset -r ScriptParentDir=$(realpath "${(Q)${0:A:h:h}}")
# Read-only name of the current script without path and extension
typeset -r ScriptBasename=${${0:A:t}%.*}
# Read-only name of the directory containing the current script
typeset -r ScriptDirname=$(basename "$(realpath "${0:A:h}")")
# Read-only extension of the current script
typeset -r ScriptExt=${0##*.}
These variables capture information about the script that called the current script, if passed as arguments. They are read-only to preserve the original calling context.
# Read-only name of the caller script (if passed as an argument)
typeset -r CallerName=$1
# Read-only directory of the caller script (if passed as an argument)
typeset -r CallerDir=$(realpath "${(Q)${1:h}}")
# Read-only absolute path of the caller script (if passed as an argument)
typeset -r CallerPath=$(realpath "$1")
These variables provide information about the current and parent processes. Since process IDs and names do not change during script execution, there is no need for the _Init
suffix, but I do set them read-only for safety.
# Read-only name of the current process
typeset -r ProcName=$(ps -o comm= -p "$$")
# Read-only name of the parent process
typeset -r ProcParentName=$(ps -o comm= -p "$PPID")
# Read-only parent process ID
typeset -r ProcParentId=$PPID
# Read-only name of the parent process
typeset -r ProcParentName=$(ps -o comm= -p $PPID)
(This section needs to be rewritten for zparams and various ways to leverage parameters in a zsh array)
These variables store the script's initial arguments as read-only, allowing derived or modified arguments to be read/write.
# Initial read-only all arguments passed to the script as an array. I use the suffix _Init to keep the original arguements, as sometimes it is useful to edit the array.
typeset -r AllArgsArrayInit=("$@")
AllArgsArray=AllArgsArrayInit
# Initial read-only first argument passed to the script
typeset -r Arg1=$1
# Initial read-only second argument passed to the script
typeset -r Arg2=$2
# Initial read-only third argument passed to the script
typeset -r Arg3=$3
These variables are related to the Git environment and can be both local and global. Local variables are specific to the current repository, while global variables are part of the user's global Git configuration.
Local Variables (set for the current repository):
# Git directory for the current repository
typeset -r GitDirLocal=$(git rev-parse --git-dir)
# Working tree of the current repository
typeset -r GitWorkTreeLocal=$(git rev-parse --show-toplevel)
# Current HEAD commit hash
typeset -r GitHeadLocal=$(git rev-parse HEAD)
Global Variables (global Git configuration):
# Path to the global Git configuration file
typeset -r GIT_CONFIG_GLOBAL=$HOME/.gitconfig
# Global Git user name
typeset -r GIT_USER_NAME_GLOBAL=$(git config --global user.name)
# Global Git user email
typeset -r GIT_USER_EMAIL_GLOBAL=$(git config --global user.email)
# LOG_FILE_FULL_PATH: Ensures the log file path is absolute, normalizing it with realpath and handling spaces correctly.
LOG_FILE_FULL_PATH=$(realpath "${(Q)${ScriptDir}/${ScriptName}.log}")
# Temporary directory: Ensures the temporary directory path is absolute and correctly handles symbolic links and spaces.
TmpDir=$(realpath "${(Q)$(mktemp -d)}")
# Counters for loops
typeset Counter=0 # when a function is called with the Counter as a parameter
local counter=0 # when all the uses of the counter are in this function's context
local _inner_counter=0 # when the counter is inside a local function.
# Status flag
typeset Success=true
# Variable to store user input
local userInput=""
# Function to log messages
log_Message() {
local message=$1
if [[ ! -f "$LOG_FILE_FULL_PATH" ]]; then
touch "$LOG_FILE_FULL_PATH" || { echo "
Error: Cannot create log file"; exit 1; }
fi
echo "$(date): $message" >> "$LOG_FILE_FULL_PATH" || { echo "Error: Cannot write to log file"; exit 1; }
}
# Main script logic
log_Message "Script started in directory: $WorkDirPath"
# Example loop with counter
for file in "$WorkDirPath"/*; do
Counter=$((Counter + 1))
log_Message "Processing file $Counter: $file"
# Example operation that might change the working directory
if [[ -d $file ]]; then
WorkDirPath=$file
log_Message "Changed working directory to: $WorkDirPath"
fi
done
log_Message "Script finished with $Counter files processed."
log_Message "Temporary files are stored in: $TmpDir"
# Clean up
rm -rf "$TmpDir"
log_Message "Temporary directory cleaned up."
# Change to a specific directory and list its contents
cd $WorkDirPath
echo "Listing contents of $WorkDirPath:"
ls
# Log a message to the log file
log_Message "This is a log entry"
# Capture existing EXIT traps
existing_trap=$(trap -p EXIT)
trap 'rm -rf "$TmpDir"; eval "$existing_trap"' EXIT
# Create a temporary directory and use it for temporary files
TmpDir=$(mktemp -d)
tmp_file=$(mktemp -p "$TmpDir")
echo "Temporary file created: $tmp_file"
# Check if the first argument is provided and modify it
if [[ -n $Arg1 ]]; then
ModifiedArg1="${Arg1}_modified"
echo "Modified argument: $ModifiedArg1"
else
echo "No first argument provided."
fi
# Assume this script is called with the caller script path as the first argument
typeset -r CallerName=$(basename "$1")
typeset -r CallerDir=$(dirname "$1")
typeset -r CallerPath=$(realpath "$1")
echo "Caller Script Name: $CallerName"
echo "Caller Script Directory: $CallerDir"
echo "Caller Script Full Path: $CallerPath"
# Check if required commands are available
required_commands=("git" "realpath" "awk")
for cmd in "${required_commands[@]}"; do
if ! command -v $cmd &> /dev/null; then
echo "Error: Required command '$cmd' is not installed."
exit 1
fi
done
# Function to handle unexpected input
validate_Input() {
local input=$1
if [[ -z "$input" ]]; then
echo "Error: Input cannot be empty."
return 1
fi
if ! [[ "$input" =~ ^[a-zA-Z0-9_-]+$ ]]; then
echo "Error: Input contains invalid characters. Only alphanumeric, underscores, and hyphens are allowed."
return 1
fi
return 0
}
# Example usage
validate_Input "$1" || exit 1
# Ensure temporary files and directories are cleaned up properly
cleanup() {
[[ -d "$TmpDir" ]] && rm -rf "$TmpDir"
echo "Cleanup completed."
}
trap cleanup EXIT
# Main script logic here
-
It can be useful to configure Finder to default to open .sh scripts with the terminal, but of course they will be executed without any parameters.
-
Some older versions of macOS don't have
realpath
. Here is a solution if you need to work on both legacy and current macOS.
if ! command -v realpath &> /dev/null; then
realpath() {
[[ $1 = /* ]] && echo "$1" || echo "$PWD/${1#./}"
}
fi
-
**Using Zsh
$fpath
for Function Libraries- Advice and demo
-
Zsh Script Template:
- Comprehensive Template for Zsh Command-Line Scripts
- Uses Zsh Common Utiilities Libraries via
$fpath
- Includes functional test scripts
- Demonstrates best practices
-
More on Avoiding Globals
- Some common overuse of global variables, and how to fix
- How to use
setop warn_create_global
effectively
-
Error Handling and Debugging:
- Techniques for error handling in Zsh scripts.
- Using
trap
for handling signals and cleaning up resources. - Debugging tools and practices (
set -x
,print -v
).
-
Advanced Parameter Expansion:
- Techniques for advanced parameter expansion and manipulation.
- Examples of using
${parameter//pattern/replacement}
and other parameter expansion features.
-
Working with Strings:
- Best practices for string manipulation in Zsh.
- Examples of string operations (concatenation, substitution, slicing).
-
Working with Files and Directories:
- Techniques for handling files and directories (checking existence, permissions).
- Examples of file and directory operations (copying, moving, deleting).
-
Function Libraries and Modularity:
- Best practices for organizing and reusing functions across scripts.
- Techniques for creating and using function libraries.
-
Testing and Continuous Integration:
- Approaches to writing tests for Zsh scripts.
- Integrating Zsh script testing into CI/CD pipelines.
-
Documentation and Comments:
- Guidelines for documenting Zsh scripts and functions.
- Best practices for writing meaningful comments.
-
Performance Optimization:
- Tips for improving the performance of Zsh scripts.
- Avoiding common pitfalls that can degrade performance.
Zsh offers several powerful but lesser-known features for debugging scripts. Here are some useful techniques:
TRAPDEBUG
Function
The TRAPDEBUG
function in Zsh can be used to execute code before every command. This can be very useful for tracing execution and debugging:
TRAPDEBUG() {
echo "Executing: $ZSH_DEBUG_CMD"
}
A more sophisticated version:
# Improved TRAPDEBUG function to trace function calls
TRAPDEBUG() {
# Disable the trap to prevent recursion
local trap_state=$TRAPDEBUG
TRAPDEBUG=''
# Log the current function and its caller if available
if [[ -n "${funcstack[2]}" ]]; then
echo "In function: ${funcstack[1]}, called from: ${funcstack[2]}"
else
echo "In function: ${funcstack[1]}"
fi
# Restore the trap
TRAPDEBUG=$trap_state
}
zsh -x
for Execution Trace
Running a script with zsh -x
will print each command before it is executed, providing a detailed execution trace:
zsh -x your_script.zsh
*set -x
and set +x
You can enable and disable execution tracing within a script using set -x
and set +x
:
set -x # Enable tracing
# Your code here
set +x # Disable tracing
*$funcstack
The $funcstack
array provides a call stack of functions. You can use it to see the sequence of function calls leading to a particular point:
echo "Function call stack:"
for ((i = 1; i <= ${#funcstack[@]}; i++)); do
echo "Caller $i: ${funcstack[$i]}"
done
Verbose Debugging
Use a verbose
flag to conditionally print debug information:
verbose=1 # Set to 1 to enable verbose mode
if [[ -n $verbose ]]; then
echo "Debugging info: variable = $variable"
fi
autoload -U zsh/zprof
Use zprof
for profiling Zsh scripts to identify performance bottlenecks:
# Enable profiling
zmodload zsh/zprof
# Your script here
# Print profiling results
zprof
DEBUG
and RETURN
Traps
You can set traps for DEBUG
and RETURN
to execute commands before and after each function:
trap 'echo "Before command: $ZSH_DEBUG_CMD"' DEBUG
trap 'echo "Function $funcstack[1] returned with status $?."' RETURN
Conditional Breakpoints with zshdb
Use zshdb
, a debugger for Zsh scripts, to set breakpoints and step through code:
zshdb your_script.zsh
Using typeset -p
Inspect Variables with typeset -p <var>
Use typeset
to inspect variables, especially associative arrays:
typeset -p my_assoc_array
Inspect all variables with typeset -p
When you run typeset -p
by itself in Zsh, it prints out the definitions of all variables, functions, and their attributes in the current shell environment. This includes simple variables, arrays, associative arrays, and functions. It's a comprehensive way to see the current state of the shell's environment.
Example Usage
Here’s how you might use typeset -p
by itself and what you can expect:
#!/bin/zsh
# Define some variables
my_var="Hello, World!"
my_array=("one" "two" "three")
typeset -A my_assoc_array
my_assoc_array[key1]="value1"
my_assoc_array[key2]="value2"
# Define a simple function
my_function() {
echo "This is my function."
}
# Print the definitions of all variables and functions
typeset -p
Output
Running typeset -p
by itself will produce output similar to the following:
typeset my_var='Hello, World!'
typeset -a my_array=( 'one' 'two' 'three' )
typeset -A my_assoc_array=( [key1]="value1" [key2]="value2" )
typeset -f my_function
Explanation
- Variables: The output includes the definitions of all the variables in the current shell environment, showing their names, types, and values.
- Arrays: Both indexed arrays and associative arrays are displayed with their elements.
- Functions: Function definitions are also included, showing their names and indicating that they are functions with
typeset -f
.
Using typeset -p
can be particularly useful for debugging and inspecting the state of your environment at various points in a script. For example, you might add typeset -p
at critical points in your script to see how variables are changing over time.
#!/bin/zsh
# Define some variables
my_var="Hello, World!"
my_array=("one" "two" "three")
typeset -A my_assoc_array
my_assoc_array[key1]="value1"
my_assoc_array[key2]="value2"
# Define a simple function
my_function() {
echo "This is my function."
}
# Initial state
echo "Initial state of the environment:"
typeset -p
# Change a variable
my_var="Goodbye, World!"
# After changing a variable
echo "State of the environment after changing a variable:"
typeset -p
The output will show the initial state of all variables and functions, followed by the state after modifying a variable:
Initial state of the environment:
typeset my_var='Hello, World!'
typeset -a my_array=( 'one' 'two' 'three' )
typeset -A my_assoc_array=( [key1]="value1" [key2]="value2" )
typeset -f my_function
State of the environment after changing a variable:
typeset my_var='Goodbye, World!'
typeset -a my_array=( 'one' 'two' 'three' )
typeset -A my_assoc_array=( [key1]="value1" [key2]="value2" )
typeset -f my_function
Conditional Command Logging
You can log commands conditionally based on certain conditions:
function log_command {
if [[ -n $verbose ]]; then
echo "Running: $@"
fi
"$@"
}
log_command ls -l
Use PS4
for Custom Debugging Prompts
Customize the debug prompt with PS4
to include more information:
export PS4='+${FUNCNAME[0]:-}:$LINENO: ${SECONDS}s: '
set -x
# Your script here
set +x
xtrace
for Specific Commands
Use xtrace
for specific commands rather than the entire script:
xtrace() {
set -x
"$@"
set +x
}
xtrace ls -l
Combining Techniques for Advanced Debugging
You can combine these techniques for more advanced debugging:
verbose=1
TRAPDEBUG() {
echo "Executing: $ZSH_DEBUG_CMD"
}
trap 'echo "Function $funcstack[1] returned with status $?."' RETURN
if [[ -n $verbose ]]; then
set -x
fi
function example_function {
local var="Hello, World!"
echo $var
}
example_function
if [[ -n $verbose ]]; then
set +x
fi
# Log the process hierarchy
echo "Process hierarchy:"
ps -o ppid,pid,command | grep -E "^ *$(ps -o ppid= -p $$) *"
# Print the definitions of all functions
echo "Function definitions:"
typeset -f
# Log the current environment variables
echo "Environment variables:"
env
# Scoped Flags for verbose and debug output
typeset -i verbose=0
typeset -i debug=0
# Initialize an scoped associative array to store trap commands
typeset -A TrapCommands
#---------------------------------------------------------------
# Function: script_Cleanup
# Description: Cleans up temporary files and resources.
# Arguments: None
# Globals:
# - TEST_FILE_PATH: Path to the temporary test key file.
#---------------------------------------------------------------
script_Cleanup() {
if [[ $debug -eq 1 ]]; then
# Print the call stack
echo "Function call stack:"
# Loop through the funcstack array from 1 to the length of funcstack
for ((i = 1; i <= ${#funcstack[@]}; i++)); do
echo "Caller $i: ${funcstack[$i]}"
done
fi
if [[ -f "$TEST_FILE_PATH" ]]; then
rm "$TEST_FILE_PATH"
if [[ $verbose -eq 1 ]]; then
echo "Test file removed."
fi
echo "Test file removed."
fi
# Reset flags
unset debug verbose
}
#---------------------------------------------------------------
# Function: add_Trap
# Description: Adds a new command to an existing trap or creates a new trap
# Arguments:
# - new_cmd: The new command to add to the trap.
# - signal: The signal for which the trap should be set.
# Globals: None
# Usage:
# add_Trap "command_to_add" "SIGNAL"
# Examples:
# add_Trap test_Script_Cleanup "EXIT"
# add_Trap test_Script_Cleanup "ERR"
#---------------------------------------------------------------
add_Trap() {
local new_Cmd=$1
local signal=$2
# Initialize the command list for the signal if it doesn't exist
if [[ -z ${TrapCommands[$signal]} ]]; then
TrapCommands[$signal]=$new_Cmd
else
TrapCommands[$signal]="${TrapCommands[$signal]}; $new_Cmd"
fi
# Set the combined trap command
trap "${TrapCommands[$signal]}" "$signal"
# Print debugging output if verbose mode is enabled
if [[ $debug -eq 1 ]]; then
echo "Adding trap for signal: $signal"
echo "New command: $new_Cmd"
echo "Combined command: ${trap_commands[$signal]}"
# Print the definition of the associative array
typeset -p trap_commands
fi
}
# Set the trap to call script_Cleanup function on EXIT and ERR for the main script
if [[ "${0##*/}" == "$_SCRIPT_TEMPLATE_TEST_SCRIPT_NAME" ]]; then
add_Trap "script_Cleanup" EXIT
fi