hsh User Manual

Copyright (C) 2010 Alexander Taler (dissent@0--0.org)
This file is part of hsh.
hsh is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
hsh is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with hsh. If not, see http://www.gnu.org/licenses/ .

Contents

  1. Introduction
    1. Installation
    2. General Usage
  2. Display Elements
    1. Input Area
    2. Session List
    3. Job Output View
    4. Job List Views
      1. Session List
      2. Running Jobs
      3. Command History
      4. Job History List
    5. Search View
    6. Key Value Views
      1. Environment
      2. Aliases
  3. Command Line Syntax
    1. Prog
      1. Aliases
      2. Builtins
    2. Glob Expansion (*,?)
    3. Environment Variables ($)
    4. Named Directories (~)
    5. Job Reference (%)
    6. \ Escapes
    7. Quoted Tokens
    8. Job Directives (#)
  4. Jobs
    1. Piped Mode
    2. Pseudo Terminal
    3. Exclusive Mode
    4. Full-Screen Mode
  5. Commands
    1. Standard Builtins
    2. Default Aliases
  6. Configuration
  7. Trouble Shooting
    1. Facilities
    2. Common Issues

Introduction

hsh is the {Happy|Human|History} shell, a command-line driven user interface to your system. It is inspired by the traditional shells (Bourne shell, C shell etc.) but replaces the scrolling terminal style output with a full-screen curses interface.

The hsh interface is split into two areas, the bottom few lines are for input, and the rest of the screen is the main area, which shows one of a number of different views, including job output, job lists, and environmental information. The cursor indicates which view has focus, and can be moved to a new view using Alt-o or Alt-Tab. The interface is dynamic, and will update automatically according to events such as changes to running jobs or user keystrokes.

Installation

hsh has a very simple manual installation. The src and doc directories may be placed anywhere on the filesystem. The src/hsh.bin script is then used to start hsh, and locates the other files relative to its location. A .hsh directory will be created in the user's home directory.

General Usage

Throughout this document keystrokes are specified using a keyname, which can be a case-sensitive letter, a single special character, like $, or a multicharacter special key name like F1 or PgUp. Additionally a C- may be prepended to indicate that the control key is held while typing the character, or a M- to indicate that the key is 'escaped' by holding the Meta or Alt key simultaneously, or typing an Escape immediately before. Input in curses is a funny thing, so some keystrokes are not available, eg. C-Tab.

The basic keystrokes which work in most views are:

C-q Quit hsh: terminate the process and jobs.
M-d Delete the current job
M-e Edit the current job. Copy its command line into the input area with a #replace directive added. When executed, this command line will replace the current job's output and command line in all job lists.
M-n Change the main view to highlight or show the next job.
M-o or M-Tab Switch focus to the next window.
M-p Change the main view to highlight or show the previous job.
M-r Rerun the current job and display its new output in place of the old.
M-s Show the search window.
M-w Toggle wrap mode, alternating between showing full lines of output text wrapped onto multiple lines, and only the beginning of each line as it can fit on the screen.
F1 or C-h Show the welcome/help page.
F2 Show the job output view.
F3 Show the list of session jobs.
F4 Show the list of running session jobs.
F5 Show the command history.
F6 Show the list of all jobs.
F10 Show the last exception view.

These standard movement and editing keys don't all apply in all circumstances. Documentation for each view may specify overrides.

C-f or Right Move forward one character along the line.
C-b or Left Move backward one character along the line.
C-p or Up Move up one line.
C-n or Down Move down one line.
C-a Move to the beginning of the line
C-e Move to the end of the line
PgUp Move up a page
PgDn Move down a page
Home Move to the top
End Move to the bottom
Backspace Delete to the left
Delete or C-d Delete to the right
C-k Delete the rest of the line
C-y Paste the most recent line deleted with C-k

Display Elements

Input Area

The input area is at the bottom of the screen, and is where you type your commands for launching jobs. By default the input area has five lines of text, with the following uses:

The cursor position is restricted to the single line of input text.

In addition to character entry and standard editing keystrokes, the following keystrokes are accepted:

Up Move back one row in history
Down Move forward one row in history
C-c Clear the currently typed command. Don't run it, but save it in history.
Tab or C-i Complete what is currently being typed as much as possible.
Enter or C-m or C-j Execute the currently typed command, creating a new job.
PgUp Move the main view up one page.
PgDn Move the main view down one page.
Home Move the main view to the top of the current job's output, or absolute top.
End Move the main view to the top of the current job's output, or absolute top.

Session List

The session list is the default view shown when a job is executed. It is similar in appearance to a traditional shell, it displays entered command lines with the job's output below it, and will show multiple if there's sufficient room in the display.

Job information is shown in blue on black or white on blue, and includes the following information:

  1. Line 1: (jobid) user@host:directory
  2. Line 2: Result> expanded command

Below each job's information its output is displayed, in white for stdout, red for stderr, and yellow for interactive stdin. The output may be hidden or truncated, in which case it will be replaced with three cyan dots.

One job is always highlighted, and its information is displayed in white on blue instead of blue on black. Several key strokes and command line structures will refer to this 'current' job.

In addition to movement keystrokes, the following keystrokes are supported:

Enter or C-m or C-j Show the job output view of the highlighted job.
Delete or Backspace or C-d Delete the current job.
M-h Hide the current job's output. This switches between three output modes: show full output, show only three lines of output, and show no output.
Job Output View

This is a main view which contains the output of a single job. The top lines contain information about the job, and the rest of the view contains the job's output. When the view has focus, the cursor keys can be used to move the visible area. The job shown in this view is synchronized with the current job in the session list.

By default the header line colours are white on blue, and they contain the following information:

  1. Line 1: context (user@host:directory) and unexpanded command
  2. Line 2: expanded command
  3. Line 3: job id, process id and state - output index - input mode - tail mode

The output area shows text in three different colours, white text is for stdout, red text is for stderr, and yellow text is for manually entered stdin (described just below). By default the top of the output is shown when the job is loaded, and the user needs to manually move the output to see the bottom. However, tail mode may be enabled to cause the output view to constantly update so that the end of the output is visible.

When a job is started, stdin is left open, and the job view may be used to manually send input to the process's stdin. Two input modes are supported, 'character' sends input to the process a character at a time, and 'line' allows editing of each line and sends input only when enter is pressed. Additionally, stdin may be closed explicitly. This is useful for interactive processes, such as shells, or those which prompt for information.

Full-screen curses applications are handled differently. For processes not run in #exclusive mode (see Jobs and JobDirective sections below) hsh recognizes that the process wants control of the terminal by identifying escape sequences in the output and marks it as being in 'full-screen' mode. When a job requiring full-screen mode receives the focus, hsh relinquishes the terminal and allows the user to interact with the process directly. You can then return to hsh by terminating the process, or typing C-z followed by Enter.

In addition to the standard movement keystrokes, the following keystrokes are recognized in the job output view:

M-i Cycle input mode. Switches between 'silent', 'line' and 'character'.
M-c Close stdin.
M-t Toggle tail mode
Enter Submit a line in 'line' input mode.

Job List Views

The job list views show a list of jobs, optionally excluding their output. There are several such views, and more may be configured if desired. The default ones are: the session list described earlier; the running jobs list which shows all jobs created in this session whose processes are still alive; the command history which shows just the commands and includes jobs which were never executed; and the job history list which shows all jobs executed in this and other sessions.

If output is not shown, a more restricted set of movement keys are supported, so that the cursor highlights a single job. The highlighted job is not the same as the job currently in the job output view.

The running jobs and full job history show the execution context, unexpanded command job id and status for each job, while the command history shows just the job id and unexpanded command.

In addition to movement keystrokes, the following keystrokes are supported:

Enter or C-m or C-j Within session list, running jobs and full job history, this shows the highlighted job in the job output view. Within command history it copies the highlighted command to the input area.
Delete or Backspace or C-d Only in session list this deletes the highlighted job.
Search View

The search view is an optional one line window between the main view and the input view. It can receive focus and accept input. The M-s key opens it, and it is closed by giving it focus and typing M-c.

Any text typed into the search view is interpreted as a regular expression and highlighted in the main view. Additionally, the pattern may be searched for in the contents of the main view.

The search view is displayed with a leading / character in white on purple to clearly identify its purpose.

PgUp Move the main view up one page.
PgDn Move the main view down one page.
Home Move the main view to the top of the current job's output, or absolute top.
End Move the main view to the top of the current job's output, or absolute top.
M-c Close the search view, and switch focus.
M-s Search forward to the next occurrence of the pattern.
M-S Search backward to the previous occurrence of the pattern.

Key Value Views

There are several data dictionaries which the user can reference and modify. Each of these shares the same interface to view and edit. Additionally there are builtins to display the view and provide an alternative for editing.

The view consists of two columns, the left contains the keys, the right the values, with a row of text for each pair. The cursor is restricted to the key and value text fields, and the Tab key is used to switch between a key and its corresponding value. To edit a field, just start typing or using other editing commands. To save the changes hit Enter, otherwise hit Tab or the up and down arrow keys to leave the field and abandon them.

Adding a new key value pair is not done using the interface directly, instead the builtin command is used. Pairs may be deleted by clearing the key field and submitting the change with Enter.

Changes that have been made, but not yet put into effect, are indicated by displaying the text in a different colour.

Environment

Access to the environment variables is provided through the env builtin described below.

Aliases

Access to defined aliases is provided through the alias builtin described below.

Command Line Syntax

hsh's command line syntax is still very young and not fully developed. I expect it to depart from that of other shells, since it will not be a programming language. Following the norms for process launching, the command line generates a list of parameters, the first being the program to execute, and the remaining ones passed to the program to use as it pleases. There are additional shell syntax tokens to complicate this.

Parameters are the coarsest level of separation on the command-line, and are separated by whitespace.

Prog - the first element

The first element of the command line specifies the program to run. If this starts with a ./ then the program will be searched for relative to the current directory. If it starts with a / then it will be searched for absolutely within the filesystem. If it starts with a + then the + is removed, and the next step of alias and builtin checking is skipped. If it is the name of an alias the alias will be expanded, and searching will begin again, skipping further alias expansion. If it is the name of a builtin, the builtin will be run. Finally, an executable will be searched for relative to all entries of the PATH environment variable.

This differs from other shells in the handling of prog names containing a /, but not starting with a / or ., eg. imagemagick/mogrify. hsh will search for such things relative to the path, while other shells will search for them relative to the current directory.

Glob Expansion (*,?,[])

Within a parameter '*', '?' and '[]' cause the text to be matched against filenames, searching from the current directory. '*' matches 0 or more characters, '?' matches any one character, '[...]' matches any one character contained between the brackets. If multiple filenames match the pattern, then it will expand to multiple parameters, and if no filenames match, it will expand to nothing. Glob characters are not legal within Prog names.

Environment Variables ($)

A '$' specifies an environment variable expansion. The name of the variable may optionally be enclosed in braces ('{' and '}'), if not it terminates on the first non-alphanumeric, non-'_' character. In the executed command this will be replaced by the value of the environment variable. This syntax is intended to duplicate that of other shells.

Named Directories (~)

A '~' specifies a named directory expansion, and is followed by the name of the directory. The name may be optionally enclosed in braces ('{' and '}'), and if not terminates on the first non-alphanumeric, non-'_' character. In the executed command this token is replaced by the user's home directory. If no name is given, the current user's home directory is used. This syntax is intended to duplicate that of other shells.

Job Reference (%)

A '%' specifies a job reference expansion, and differs greatly from other shells. It can be used to access the output, input and pid of the job. The reference has the following syntax:

  '%' ['{'] [ job-id ] job-field [ ',' job-field ]* ['}']

job-id: optional numerical job id, defaulting to the currently visible job.
job-field: specifies what to expand. Multiple job-fields may be specified separated by commas, which will result in an expansion to multiple parameters. Possible values of job-field are:

Some typical examples:

kill %p
kill %668p
Send a TERM signal to the currently visible job, if it's running. Then do the same for the process of job 668.
ls src/hsh
egrep -v '\.pyc$' %fo
Look at the directory listing, then limit output to files not ending with '.pyc'.
sed 's/^\([^:]*\):.*/\1/' /etc/passwd
finger %to
Get login information for all system users.

\ Escapes

The \ character can be used to 'escape' the next character, indicating that it should be taken literally and not subject to any special interpretation by hsh, including any expansions, or parameter splitting in the case of whitespace.

Quoted Tokens

Two types of quoted tokens are supported, which are useful to simplify entry when multiple characters must be entered without special interpretation. Text surrounded by double quotes ('"') will not be split on whitespace into multiple parameters. Text surrounded by single quotes (''') will not be split on whitespace, and additionally will not be subject to any expansion.

Job Directives (#)

A '#' starts a special directive for launching jobs. These are interpreted by hsh itself, and do not result in any expansion on the command line. A job directive can only start at the beginning of a parameter, #'s in the middle of tokens or within quotes have no special meaning to hsh. The token consists of all characters following the # until the next whitespace. Some of these directives are explained in more detail in the Jobs section. Some directives come in groups from which at most one option may be selected, in which case the last specified directive takes precedence.

Some groups of directives are mutually incompatible, and only the last one of the group will be used: the first are the job I/O directives: '#piped', '#exclusive', '#pty'; the second are the visibility directives: '#changeview, '#dontchangeview.

The following Job Directives are defined:

Jobs

When you type a command and hit Enter, hsh creates a 'Job'. Each Job consists of details about the command line used to create it, as well as some optional stuff like an external program, process, output and input. External programs can be quite finicky about the environment they run in, so hsh must deal with them appropriately. There are still some rough edges to work out in this area.

Piped Mode

By default jobs use piped mode, where programs are run with pipes attached to standard input, output and error. This mode is appropriate for most non-interactive programs, and some interactive ones. It allows hsh the greatest freedom for handling the external processes, without consumption of system resources. Interactive programs may wish to use pseudo terminals or exclusive mode described below.

Piped mode may be selected explicitly using the '#piped' job directive.

Pseudo Terminal

Programs which require a terminal may be run within a pseudo terminal. In this mode the program gets access to a terminal interface on its standard input and output, but hsh controls the other end of the pseudo terminal, and acts as an intermediary to the actual terminal. This allows the output and input of the program to be stored by hsh.

Pseudo terminals are not available on all platforms, and are not the default because they consume potentially scarce system resources.

To run a job in exclusive mode, use the '#pty' job directive.

Exclusive Mode

Exclusive mode is available for the most demanding programs, granting them full control over the terminal. This is essentially the same way that a traditional shell executes all of its processes. It is appropriate for jobs that require terminal access, and which do not work within a pseudo terminal.

Output of jobs run in exclusive mode is not stored by hsh and cannot be processed like output of other types of jobs.

To run a job in exclusive mode, use the '#exclusive' job directive.

Exclusive mode is useful for programs that write directly to the terminal, notably security-minded programs like 'ssh', 'sudo' and 'gpg'. Additionally, some curses programs may be more manageable in exclusive mode than in a psuedo-terminal.

As in job control for regular shells, a Ctrl-Z character will typically send a STOP signal to the foreground process. Hsh will recognize that the job has stopped, resume itself and reclaim control of the terminal.

Full-Screen Mode

Programs which are running in piped or pseudo terminal mode and send terminal control codes are automatically put into full-screen mode, allowing the use of curses interfaces. When focus is given to a job in full-screen mode, hsh relinquishes control over the terminal and allows the program to freely control the terminal. However, hsh is still acting as intermediary, which introduces some limitations, and some benefits.

Jobs in exclusive mode are not being mediated by hsh and so will not use full-screen mode. Unlike exclusive mode, full-screen mode does not require the user (or an alias) to explicitly enable it, instead it is automatically enabled for any program which uses terminal codes.

Programs in full-screen mode are not able to determine the actual terminal size, and communication of extended keystrokes may not work fully as expected. Output is not recorded in full-screen mode, and consequently can't be processed as for other jobs.

The Ctrl-Z character is interpreted specially in full-screen mode and is trapped by hsh instead of being sent through to the job. If the next character is another Ctrl-Z then a single Ctrl-Z will be sent to the job, an action which will often result in suspending the job's execution. If the next character is Enter (newline/Ctrl-M/Ctrl-J), then hsh will move focus away from the job, but without suspending it.

Commands

Standard Builtins

alias With no arguments, show the alias management view; with one argument, delete the named alias; with two arguments, set the alias named by the first argument to the value of the second argument, creating it if it doesn't exist.
cause_internal_error Raise an exception, useful if you want to see the exception view.
cd Change the working directory to the named directory, or home directory if no arguments are given.
copying Display the terms for copying hsh, and the license under which it is distributed.
debug_dump Evaluate the argument as Python code and write the results to the logfile. Intended for development and tricky debugging. Definitely could do with some improvements.
env With no arguments, show the environment management view; with one argument, delete the named environment variable; with two arguments, set the environment variable named by the first argument to the value of the second argument, creating it if it doesn't exist.
exit Exit the hsh process.
hsh_view With no arguments, show a list of all the available views; with one argument, show the named view in the main area.
reexec Restart the hsh process, reloading any Python code. This is mostly useful for development, and needs improvement to preserve more state.
version Display the version of hsh this is.

Default Aliases

AliasValueDetails
gpg gpg #exclusive For security, and because gpg can trash the terminal.
sudo sudo #exclusive For security, and because sudo can trash the terminal.
ssh ssh #exclusive For security, and because ssh can trash the terminal.
aptitude aptitude #exclusive Curses app requiring full screen access.
info info #exclusive Curses app requiring full screen access.
python python #pty Python won't go interactive without a tty.
sh sh #pty sh won't go interactive without a tty.
top top #pty top insists on having access to a tty.
quit exit quit is a familiar way of terminating a shell.
logout exit logout is a familiar way of terminating a shell.
bye exit bye is a cute way of terminating a shell.
history hsh_view CommandHistory Traditional shell built-in to display command history.
jobs hsh_view RunningJobs Traditional shell built-in to display a list of running jobs .

Configuration

The user level configuration mechanism has not yet been defined. Keybind lists and visual configuration files exist in the source code, and the user configuration will follow this mechanism. It is acceptable to modify these files currently in the installation, although this will create difficulty in managing the source. This area is high priority to address.

The source code files of interest are hsh/curses_display/keybinds.py and hsh/config.py.

It is expected that many configurable options will be added.

Trouble Shooting

Facilities

hsh provides several facilities for tracking down problems:

Unusual events or user errors can trigger an alert message on the bottom line of the display.

Exceptions in the main thread are caught and displayed in the Exception View. hsh continues to run after the exception, although its runtime integrity may be compromised. The exception view is available on F10 by default. Try running cause_internal_error for a harmless look at the exception view.

A log file is created as ~/.hsh/debug.log, which includes various debugging information, most importantly the stack traces of any exceptions.

The debug_dump builtin can be used to examine internal data structures using python code directly. (This facility could do with some improvement.)

Common Issues

Sometimes (particularly during development) a job's output will stop appearing. This is typically due to an error in the job's output processing thread. Take a look in the debug.log file for any exception traces from the processing thread. (This issue really needs an alert message.)