hsh User Manual

Contents

1. Introduction
   1. Installation
   2. General Usage
2. Display Elements
   1. Input Area
   2. Session List
   3. Job Output View
   4. Job List Views
      1. Session List
      2. Running Jobs
      3. Command History
      4. Job History List
   5. Search View
   6. Key Value Views
      1. Environment
      2. Aliases
3. Command Line Syntax
   1. Prog
      1. Aliases
      2. Builtins
   2. Glob Expansion (*,?)
   3. Environment Variables ($)
   4. Named Directories (~)
   5. Job Reference (%)
   6. \ Escapes
   7. Quoted Tokens
   8. Job Directives (#)
4. Jobs
   1. Piped Mode
   2. Pseudo Terminal
   3. Exclusive Mode
   4. Full-Screen Mode
5. Commands
   1. Standard Builtins
   2. Default Aliases
6. Configuration
7. Trouble Shooting
   1. Facilities
   2. Common Issues

Introduction

hsh is the {Happy|Human|History} shell, a command-line driven user interface to your system. It is inspired by the traditional shells (Bourne shell, C shell etc.) but replaces the scrolling terminal style output with a full-screen curses interface.

The hsh interface is split into two areas, the bottom few lines are for input, and the rest of the screen is the main area, which shows one of a number of different views, including job output, job lists, and environmental information. The cursor indicates which view has focus, and can be moved to a new view using Alt-o or Alt-Tab. The interface is dynamic, and will update automatically according to events such as changes to running jobs or user keystrokes.

Installation

hsh has a very simple manual installation. The src and doc directories may be placed anywhere on the filesystem. The src/hsh.bin script is then used to start hsh, and locates the other files relative to its location. A .hsh directory will be created in the user's home directory.

General Usage

Throughout this document keystrokes are specified using a keyname, which can be a case-sensitive letter, a single special character, like $, or a multicharacter special key name like F1 or PgUp. Additionally a C- may be prepended to indicate that the control key is held while typing the character, or a M- to indicate that the key is 'escaped' by holding the Meta or Alt key simultaneously, or typing an Escape immediately before. Input in curses is a funny thing, so some keystrokes are not available, eg. C-Tab.

The basic keystrokes which work in most views are:

These standard movement and editing keys don't all apply in all circumstances. Documentation for each view may specify overrides.

Display Elements

Input Area

The input area is at the bottom of the screen, and is where you type your commands for launching jobs. By default the input area has five lines of text, with the following uses:

• Line 1 shows the execution context: user @ hostname : current directory
• Line 2 shows the fully expanded command as it will be executed.
• Line 3 shows possible completions for text currently being entered.
• Line 4 shows the raw text as entered by the user, and the cursor position.
• Line 5 is a message line to alert the user of problems.

The cursor position is restricted to the single line of input text.

In addition to character entry and standard editing keystrokes, the following keystrokes are accepted:

Session List

The session list is the default view shown when a job is executed. It is similar in appearance to a traditional shell, it displays entered command lines with the job's output below it, and will show multiple if there's sufficient room in the display.

Job information is shown in blue on black or white on blue, and includes the following information:

1. Line 1: (jobid) user@host:directory
2. Line 2: Result> expanded command

Below each job's information its output is displayed, in white for stdout, red for stderr, and yellow for interactive stdin. The output may be hidden or truncated, in which case it will be replaced with three cyan dots.

One job is always highlighted, and its information is displayed in white on blue instead of blue on black. Several key strokes and command line structures will refer to this 'current' job.

In addition to movement keystrokes, the following keystrokes are supported:

This is a main view which contains the output of a single job. The top lines contain information about the job, and the rest of the view contains the job's output. When the view has focus, the cursor keys can be used to move the visible area. The job shown in this view is synchronized with the current job in the session list.

By default the header line colours are white on blue, and they contain the following information:

1. Line 1: context (user@host:directory) and unexpanded command
2. Line 2: expanded command
3. Line 3: job id, process id and state - output index - input mode - tail mode

The output area shows text in three different colours, white text is for stdout, red text is for stderr, and yellow text is for manually entered stdin (described just below). By default the top of the output is shown when the job is loaded, and the user needs to manually move the output to see the bottom. However, tail mode may be enabled to cause the output view to constantly update so that the end of the output is visible.

When a job is started, stdin is left open, and the job view may be used to manually send input to the process's stdin. Two input modes are supported, 'character' sends input to the process a character at a time, and 'line' allows editing of each line and sends input only when enter is pressed. Additionally, stdin may be closed explicitly. This is useful for interactive processes, such as shells, or those which prompt for information.

Full-screen curses applications are handled differently. For processes not run in #exclusive mode (see Jobs and JobDirective sections below) hsh recognizes that the process wants control of the terminal by identifying escape sequences in the output and marks it as being in 'full-screen' mode. When a job requiring full-screen mode receives the focus, hsh relinquishes the terminal and allows the user to interact with the process directly. You can then return to hsh by terminating the process, or typing C-z followed by Enter.

In addition to the standard movement keystrokes, the following keystrokes are recognized in the job output view:

Job List Views

The job list views show a list of jobs, optionally excluding their output. There are several such views, and more may be configured if desired. The default ones are: the session list described earlier; the running jobs list which shows all jobs created in this session whose processes are still alive; the command history which shows just the commands and includes jobs which were never executed; and the job history list which shows all jobs executed in this and other sessions.

If output is not shown, a more restricted set of movement keys are supported, so that the cursor highlights a single job. The highlighted job is not the same as the job currently in the job output view.

The running jobs and full job history show the execution context, unexpanded command job id and status for each job, while the command history shows just the job id and unexpanded command.

In addition to movement keystrokes, the following keystrokes are supported:

The search view is an optional one line window between the main view and the input view. It can receive focus and accept input. The M-s key opens it, and it is closed by giving it focus and typing M-c.

Any text typed into the search view is interpreted as a regular expression and highlighted in the main view. Additionally, the pattern may be searched for in the contents of the main view.

The search view is displayed with a leading / character in white on purple to clearly identify its purpose.

Key Value Views

There are several data dictionaries which the user can reference and modify. Each of these shares the same interface to view and edit. Additionally there are builtins to display the view and provide an alternative for editing.

The view consists of two columns, the left contains the keys, the right the values, with a row of text for each pair. The cursor is restricted to the key and value text fields, and the Tab key is used to switch between a key and its corresponding value. To edit a field, just start typing or using other editing commands. To save the changes hit Enter, otherwise hit Tab or the up and down arrow keys to leave the field and abandon them.

Adding a new key value pair is not done using the interface directly, instead the builtin command is used. Pairs may be deleted by clearing the key field and submitting the change with Enter.

Changes that have been made, but not yet put into effect, are indicated by displaying the text in a different colour.

Access to the environment variables is provided through the env builtin described below.

Access to defined aliases is provided through the alias builtin described below.

Command Line Syntax

hsh's command line syntax is still very young and not fully developed. I expect it to depart from that of other shells, since it will not be a programming language. Following the norms for process launching, the command line generates a list of parameters, the first being the program to execute, and the remaining ones passed to the program to use as it pleases. There are additional shell syntax tokens to complicate this.

Parameters are the coarsest level of separation on the command-line, and are separated by whitespace.

Prog - the first element

The first element of the command line specifies the program to run. If this starts with a ./ then the program will be searched for relative to the current directory. If it starts with a / then it will be searched for absolutely within the filesystem. If it starts with a + then the + is removed, and the next step of alias and builtin checking is skipped. If it is the name of an alias the alias will be expanded, and searching will begin again, skipping further alias expansion. If it is the name of a builtin, the builtin will be run. Finally, an executable will be searched for relative to all entries of the PATH environment variable.

This differs from other shells in the handling of prog names containing a /, but not starting with a / or ., eg. imagemagick/mogrify. hsh will search for such things relative to the path, while other shells will search for them relative to the current directory.

Glob Expansion (*,?,[])

Within a parameter '*', '?' and '[]' cause the text to be matched against filenames, searching from the current directory. '*' matches 0 or more characters, '?' matches any one character, '[...]' matches any one character contained between the brackets. If multiple filenames match the pattern, then it will expand to multiple parameters, and if no filenames match, it will expand to nothing. Glob characters are not legal within Prog names.

Environment Variables ($)

A '$' specifies an environment variable expansion. The name of the variable may optionally be enclosed in braces ('{' and '}'), if not it terminates on the first non-alphanumeric, non-'_' character. In the executed command this will be replaced by the value of the environment variable. This syntax is intended to duplicate that of other shells.

Named Directories (~)

A '~' specifies a named directory expansion, and is followed by the name of the directory. The name may be optionally enclosed in braces ('{' and '}'), and if not terminates on the first non-alphanumeric, non-'_' character. In the executed command this token is replaced by the user's home directory. If no name is given, the current user's home directory is used. This syntax is intended to duplicate that of other shells.

Job Reference (%)

A '%' specifies a job reference expansion, and differs greatly from other shells. It can be used to access the output, input and pid of the job. The reference has the following syntax:

  '%' ['{'] [ job-id ] job-field [ ',' job-field ]* ['}']

job-id: optional numerical job id, defaulting to the currently visible job.
job-field: specifies what to expand. Multiple job-fields may be specified separated by commas, which will result in an expansion to multiple parameters. Possible values of job-field are:

• 'p': Expands to the process id of a running job.
• 'fo': Expands to the name of a file containing the job's standard output.
• 'fe': Expands to the name of a file containing the job's standard error.
• 'fi': Expands to the name of a file containing the job's standard input.
• 'to': Expands to the text of the job's standard output.
• 'te': Expands to the text of the job's standard error.
• 'ti': Expands to the text of the job's standard input.

Some typical examples:

\ Escapes

The \ character can be used to 'escape' the next character, indicating that it should be taken literally and not subject to any special interpretation by hsh, including any expansions, or parameter splitting in the case of whitespace.

Quoted Tokens

Two types of quoted tokens are supported, which are useful to simplify entry when multiple characters must be entered without special interpretation. Text surrounded by double quotes ('"') will not be split on whitespace into multiple parameters. Text surrounded by single quotes (''') will not be split on whitespace, and additionally will not be subject to any expansion.

Job Directives (#)

A '#' starts a special directive for launching jobs. These are interpreted by hsh itself, and do not result in any expansion on the command line. A job directive can only start at the beginning of a parameter, #'s in the middle of tokens or within quotes have no special meaning to hsh. The token consists of all characters following the # until the next whitespace. Some of these directives are explained in more detail in the Jobs section. Some directives come in groups from which at most one option may be selected, in which case the last specified directive takes precedence.

Some groups of directives are mutually incompatible, and only the last one of the group will be used: the first are the job I/O directives: '#piped', '#exclusive', '#pty'; the second are the visibility directives: '#changeview, '#dontchangeview.

The following Job Directives are defined:

• "#exclusive": Launch job in exclusive mode.
• "#piped": Launch job in piped mode, which is the default.
• "#pty": Launch job in a pseudo terminal.

• "#changeview": Switch to the session list view, the default behaviour.
• "#dontchangeview": Don't switch to the session list.

• "#replace": This launched job will replace the job with the given id in all contexts, discarding the previous command line and output.

Jobs

When you type a command and hit Enter, hsh creates a 'Job'. Each Job consists of details about the command line used to create it, as well as some optional stuff like an external program, process, output and input. External programs can be quite finicky about the environment they run in, so hsh must deal with them appropriately. There are still some rough edges to work out in this area.

Piped Mode

By default jobs use piped mode, where programs are run with pipes attached to standard input, output and error. This mode is appropriate for most non-interactive programs, and some interactive ones. It allows hsh the greatest freedom for handling the external processes, without consumption of system resources. Interactive programs may wish to use pseudo terminals or exclusive mode described below.

Piped mode may be selected explicitly using the '#piped' job directive.

Pseudo Terminal

Programs which require a terminal may be run within a pseudo terminal. In this mode the program gets access to a terminal interface on its standard input and output, but hsh controls the other end of the pseudo terminal, and acts as an intermediary to the actual terminal. This allows the output and input of the program to be stored by hsh.

Pseudo terminals are not available on all platforms, and are not the default because they consume potentially scarce system resources.

To run a job in exclusive mode, use the '#pty' job directive.

Exclusive Mode

Exclusive mode is available for the most demanding programs, granting them full control over the terminal. This is essentially the same way that a traditional shell executes all of its processes. It is appropriate for jobs that require terminal access, and which do not work within a pseudo terminal.

Output of jobs run in exclusive mode is not stored by hsh and cannot be processed like output of other types of jobs.

To run a job in exclusive mode, use the '#exclusive' job directive.

Exclusive mode is useful for programs that write directly to the terminal, notably security-minded programs like 'ssh', 'sudo' and 'gpg'. Additionally, some curses programs may be more manageable in exclusive mode than in a psuedo-terminal.

As in job control for regular shells, a Ctrl-Z character will typically send a STOP signal to the foreground process. Hsh will recognize that the job has stopped, resume itself and reclaim control of the terminal.

Full-Screen Mode

Programs which are running in piped or pseudo terminal mode and send terminal control codes are automatically put into full-screen mode, allowing the use of curses interfaces. When focus is given to a job in full-screen mode, hsh relinquishes control over the terminal and allows the program to freely control the terminal. However, hsh is still acting as intermediary, which introduces some limitations, and some benefits.

Jobs in exclusive mode are not being mediated by hsh and so will not use full-screen mode. Unlike exclusive mode, full-screen mode does not require the user (or an alias) to explicitly enable it, instead it is automatically enabled for any program which uses terminal codes.

Programs in full-screen mode are not able to determine the actual terminal size, and communication of extended keystrokes may not work fully as expected. Output is not recorded in full-screen mode, and consequently can't be processed as for other jobs.

The Ctrl-Z character is interpreted specially in full-screen mode and is trapped by hsh instead of being sent through to the job. If the next character is another Ctrl-Z then a single Ctrl-Z will be sent to the job, an action which will often result in suspending the job's execution. If the next character is Enter (newline/Ctrl-M/Ctrl-J), then hsh will move focus away from the job, but without suspending it.

Commands

Standard Builtins

Default Aliases

Configuration

The user level configuration mechanism has not yet been defined. Keybind lists and visual configuration files exist in the source code, and the user configuration will follow this mechanism. It is acceptable to modify these files currently in the installation, although this will create difficulty in managing the source. This area is high priority to address.

The source code files of interest are hsh/curses_display/keybinds.py and hsh/config.py.

It is expected that many configurable options will be added.

Trouble Shooting

Facilities

hsh provides several facilities for tracking down problems:

Unusual events or user errors can trigger an alert message on the bottom line of the display.

Exceptions in the main thread are caught and displayed in the Exception View. hsh continues to run after the exception, although its runtime integrity may be compromised. The exception view is available on F10 by default. Try running cause_internal_error for a harmless look at the exception view.

A log file is created as ~/.hsh/debug.log, which includes various debugging information, most importantly the stack traces of any exceptions.

The debug_dump builtin can be used to examine internal data structures using python code directly. (This facility could do with some improvement.)

Common Issues

Sometimes (particularly during development) a job's output will stop appearing. This is typically due to an error in the job's output processing thread. Take a look in the debug.log file for any exception traces from the processing thread. (This issue really needs an alert message.)