BASSHFS 1.0

master
Paul Kolano 2019-10-08 18:17:22 -07:00
commit 4c98d6c14b
4 changed files with 2167 additions and 0 deletions

272
COPYING 100644
View File

@ -0,0 +1,272 @@
NASA OPEN SOURCE AGREEMENT VERSION 1.3
THIS OPEN SOURCE AGREEMENT ("AGREEMENT") DEFINES THE RIGHTS OF
USE, REPRODUCTION, DISTRIBUTION, MODIFICATION AND
REDISTRIBUTION OF CERTAIN COMPUTER SOFTWARE ORIGINALLY
RELEASED BY THE UNITED STATES GOVERNMENT AS REPRESENTED BY
THE GOVERNMENT AGENCY LISTED BELOW ("GOVERNMENT AGENCY").
THE UNITED STATES GOVERNMENT, AS REPRESENTED BY GOVERNMENT
AGENCY, IS AN INTENDED THIRD-PARTY BENEFICIARY OF ALL
SUBSEQUENT DISTRIBUTIONS OR REDISTRIBUTIONS OF THE SUBJECT
SOFTWARE. ANYONE WHO USES, REPRODUCES, DISTRIBUTES, MODIFIES
OR REDISTRIBUTES THE SUBJECT SOFTWARE, AS DEFINED HEREIN, OR
ANY PART THEREOF, IS, BY THAT ACTION, ACCEPTING IN FULL THE
RESPONSIBILITIES AND OBLIGATIONS CONTAINED IN THIS AGREEMENT.
Government Agency: NASA Ames Research Center
Government Agency Original Software Designation: NASA Ames Research Center
Government Agency Original Software Title: BASSHFS: Bash-Accessible SSH File System
User Registration Requested. Please e-mail Government Agency Point of Contact
Government Agency Point of Contact for Original Software: Paul Kolano
1. DEFINITIONS
A. "Contributor" means Government Agency, as the developer of the
Original Software, and any entity that makes a Modification.
B. "Covered Patents" mean patent claims licensable by a Contributor that
are necessarily infringed by the use or sale of its Modification alone or
when combined with the Subject Software.
C. "Display" means the showing of a copy of the Subject Software,
either directly or by means of an image, or any other device.
D. "Distribution" means conveyance or transfer of the Subject Software,
regardless of means, to another.
E. "Larger Work" means computer software that combines Subject
Software, or portions thereof, with software separate from the Subject
Software that is not governed by the terms of this Agreement.
F. "Modification" means any alteration of, including addition to or
deletion from, the substance or structure of either the Original
Software or Subject Software, and includes derivative works, as that
term is defined in the Copyright Statute, 17 USC 101. However, the
act of including Subject Software as part of a Larger Work does not in
and of itself constitute a Modification.
G. "Original Software" means the computer software first released under
this Agreement by Government Agency with Government Agency
designation NASA Ames Research Center and entitled
BASSHFS: Bash-Accessible SSH File System, including source code,
object code and accompanying documentation, if any.
H. "Recipient" means anyone who acquires the Subject Software under
this Agreement, including all Contributors.
I. "Redistribution" means Distribution of the Subject Software after a
Modification has been made.
J. "Reproduction" means the making of a counterpart, image or copy of
the Subject Software.
K. "Sale" means the exchange of the Subject Software for money or
equivalent value.
L. "Subject Software" means the Original Software, Modifications, or
any respective parts thereof.
M. "Use" means the application or employment of the Subject Software
for any purpose.
2. GRANT OF RIGHTS
A. Under Non-Patent Rights: Subject to the terms and conditions of this
Agreement, each Contributor, with respect to its own contribution to
the Subject Software, hereby grants to each Recipient a non-exclusive,
world-wide, royalty-free license to engage in the following activities
pertaining to the Subject Software:
1. Use
2. Distribution
3. Reproduction
4. Modification
5. Redistribution
6. Display
B. Under Patent Rights: Subject to the terms and conditions of this
Agreement, each Contributor, with respect to its own contribution to
the Subject Software, hereby grants to each Recipient under Covered
Patents a non-exclusive, world-wide, royalty-free license to engage in
the following activities pertaining to the Subject Software:
1. Use
2. Distribution
3. Reproduction
4. Sale
5. Offer for Sale
C. The rights granted under Paragraph B. also apply to the combination of
a Contributor's Modification and the Subject Software if, at the time
the Modification is added by the Contributor, the addition of such
Modification causes the combination to be covered by the Covered
Patents. It does not apply to any other combinations that include a
Modification.
D. The rights granted in Paragraphs A. and B. allow the Recipient to
sublicense those same rights. Such sublicense must be under the same
terms and conditions of this Agreement.
3. OBLIGATIONS OF RECIPIENT
A. Distribution or Redistribution of the Subject Software must be made
under this Agreement except for additions covered under paragraph
3H.
1. Whenever a Recipient distributes or redistributes the Subject
Software, a copy of this Agreement must be included with each
copy of the Subject Software; and
2. If Recipient distributes or redistributes the Subject Software in
any form other than source code, Recipient must also make the
source code freely available, and must provide with each copy of
the Subject Software information on how to obtain the source
code in a reasonable manner on or through a medium
customarily used for software exchange.
B. Each Recipient must ensure that the following copyright notice
appears prominently in the Subject Software:
BASSHFS: BashAccessible SSH File System
Copyright (C) 2019 United States Government as represented by the
Administrator of the National Aeronautics and Space Administration.
All Rights Reserved.
BASSHFS: Bash-Accessible SSH File System contains 3rd party code.
Copyright notices for 3rd party code below. The license can be found
at https://dev.perl.org/licenses/artistic.html.
Perl module Net::SFTP::Foreign
Copyright (c) 2005-2019 Salvador Fandino (sfandino@yahoo.com).
Copyright (c) 2001 Benjamin Trott, Copyright (c) 2003 David Rolsky.
_glob_to_regex method based on code (c) 2002 Richard Clamp.
C. Each Contributor must characterize its alteration of the Subject
Software as a Modification and must identify itself as the originator of
its Modification in a manner that reasonably allows subsequent
Recipients to identify the originator of the Modification. In fulfillment
of these requirements, Contributor must include a file (e.g., a change
log file) that describes the alterations made and the date of the
alterations, identifies Contributor as originator of the alterations, and
consents to characterization of the alterations as a Modification, for
example, by including a statement that the Modification is derived,
directly or indirectly, from Original Software provided by Government
Agency. Once consent is granted, it may not thereafter be revoked.
D. A Contributor may add its own copyright notice to the Subject
Software. Once a copyright notice has been added to the Subject
Software, a Recipient may not remove it without the express
permission of the Contributor who added the notice.
E. A Recipient may not make any representation in the Subject Software
or in any promotional, advertising or other material that may be
construed as an endorsement by Government Agency or by any prior
Recipient of any product or service provided by Recipient, or that may
seek to obtain commercial advantage by the fact of Government
Agency's or a prior Recipient's participation in this Agreement.
F. In an effort to track usage and maintain accurate records of the Subject
Software, each Recipient, upon receipt of the Subject Software, is
requested to provide Government Agency, by e-mail to the
Government Agency Point of Contact listed in clause 5.F., the
following information: name and email. Recipient's name and
personal information shall be used for statistical purposes only. Once a
Recipient makes a Modification available, it is requested that the
Recipient inform Government Agency, by e-mail to the Government
Agency Point of Contact listed in clause 5.F., how to access the
Modification.
G. Each Contributor represents that that its Modification is believed to be
Contributor's original creation and does not violate any existing
agreements, regulations, statutes or rules, and further that Contributor
has sufficient rights to grant the rights conveyed by this Agreement.
H. A Recipient may choose to offer, and to charge a fee for, warranty,
support, indemnity and/or liability obligations to one or more other
Recipients of the Subject Software. A Recipient may do so, however,
only on its own behalf and not on behalf of Government Agency or
any other Recipient. Such a Recipient must make it absolutely clear
that any such warranty, support, indemnity and/or liability obligation
is offered by that Recipient alone. Further, such Recipient agrees to
indemnify Government Agency and every other Recipient for any
liability incurred by them as a result of warranty, support, indemnity
and/or liability offered by such Recipient.
I. A Recipient may create a Larger Work by combining Subject Software
with separate software not governed by the terms of this agreement
and distribute the Larger Work as a single product. In such case, the
Recipient must make sure Subject Software, or portions thereof,
included in the Larger Work is subject to this Agreement.
J. Notwithstanding any provisions contained herein, Recipient is hereby
put on notice that export of any goods or technical data from the
United States may require some form of export license from the U.S.
Government. Failure to obtain necessary export licenses may result in
criminal liability under U.S. laws. Government Agency neither
represents that a license shall not be required nor that, if required, it
shall be issued. Nothing granted herein provides any such export
license.
4. DISCLAIMER OF WARRANTIES AND LIABILITIES; WAIVER AND
INDEMNIFICATION
A. No Warranty: THE SUBJECT SOFTWARE IS PROVIDED "AS IS"
WITHOUT ANY WARRANTY OF ANY KIND, EITHER
EXPRESSED, IMPLIED, OR STATUTORY, INCLUDING, BUT
NOT LIMITED TO, ANY WARRANTY THAT THE SUBJECT
SOFTWARE WILL CONFORM TO SPECIFICATIONS, ANY
IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE, OR FREEDOM FROM
INFRINGEMENT, ANY WARRANTY THAT THE SUBJECT
SOFTWARE WILL BE ERROR FREE, OR ANY WARRANTY
THAT DOCUMENTATION, IF PROVIDED, WILL CONFORM TO
THE SUBJECT SOFTWARE. THIS AGREEMENT DOES NOT, IN
ANY MANNER, CONSTITUTE AN ENDORSEMENT BY
GOVERNMENT AGENCY OR ANY PRIOR RECIPIENT OF ANY
RESULTS, RESULTING DESIGNS, HARDWARE, SOFTWARE
PRODUCTS OR ANY OTHER APPLICATIONS RESULTING
FROM USE OF THE SUBJECT SOFTWARE. FURTHER,
GOVERNMENT AGENCY DISCLAIMS ALL WARRANTIES AND
LIABILITIES REGARDING THIRD-PARTY SOFTWARE, IF
PRESENT IN THE ORIGINAL SOFTWARE, AND DISTRIBUTES
IT "AS IS."
B. Waiver and Indemnity: RECIPIENT AGREES TO WAIVE ANY
AND ALL CLAIMS AGAINST THE UNITED STATES
GOVERNMENT, ITS CONTRACTORS AND
SUBCONTRACTORS, AS WELL AS ANY PRIOR RECIPIENT. IF
RECIPIENT'S USE OF THE SUBJECT SOFTWARE RESULTS IN
ANY LIABILITIES, DEMANDS, DAMAGES, EXPENSES OR
LOSSES ARISING FROM SUCH USE, INCLUDING ANY
DAMAGES FROM PRODUCTS BASED ON, OR RESULTING
FROM, RECIPIENT'S USE OF THE SUBJECT SOFTWARE,
RECIPIENT SHALL INDEMNIFY AND HOLD HARMLESS THE
UNITED STATES GOVERNMENT, ITS CONTRACTORS AND
SUBCONTRACTORS, AS WELL AS ANY PRIOR RECIPIENT, TO
THE EXTENT PERMITTED BY LAW. RECIPIENT'S SOLE
REMEDY FOR ANY SUCH MATTER SHALL BE THE
IMMEDIATE, UNILATERAL TERMINATION OF THIS
AGREEMENT.
5. GENERAL TERMS
A. Termination: This Agreement and the rights granted hereunder will
terminate automatically if a Recipient fails to comply with these terms
and conditions, and fails to cure such noncompliance within thirty (30)
days of becoming aware of such noncompliance. Upon termination, a
Recipient agrees to immediately cease use and distribution of the
Subject Software. All sublicenses to the Subject Software properly
granted by the breaching Recipient shall survive any such termination
of this Agreement.
B. Severability: If any provision of this Agreement is invalid or
unenforceable under applicable law, it shall not affect the validity or
enforceability of the remainder of the terms of this Agreement.
C. Applicable Law: This Agreement shall be subject to United States
federal law only for all purposes, including, but not limited to,
determining the validity of this Agreement, the meaning of its
provisions and the rights, obligations and remedies of the parties.
D. Entire Understanding: This Agreement constitutes the entire
understanding and agreement of the parties relating to release of the
Subject Software and may not be superseded, modified or amended
except by further written agreement duly executed by the parties.
E. Binding Authority: By accepting and using the Subject Software
under this Agreement, a Recipient affirms its authority to bind the
Recipient to all terms and conditions of this Agreement and that that
Recipient hereby agrees to all terms and conditions herein.
F. Point of Contact: Any Recipient contact with Government Agency is
to be directed to the designated representative as follows:
Paul Kolano
Paul.Kolano@nasa.gov

36
README.md 100644
View File

@ -0,0 +1,36 @@
Bash-Accessible SSH File System (BASSHFS)
=========================================
Working with remote systems over SSH is common in HPC environments
where the size of data sets makes them nontrivial to relocate. To run
arbitrary commands on that data, a full SSH session is required. There
are cases, however, when the user may wish to perform simpler operations
such as checking file existence and size, viewing differences between
configuration files, creating directories, etc. that can be achieved
with more limited access. Juggling multiple sessions to multiple hosts
may be inconvenient for such simple tasks. BASSHFS is a tool that
allows users to perform such tasks within a single terminal on a single
host by transparently carrying out remote operations as needed to
present remote files as if they are locally mounted when using the bash
shell.
BASSHFS is similar to the existing SSHFS utility except it is does not
require FUSE kernel support. Instead, BASSHFS uses the aliasing and
function mechanisms of the bash shell to intercept program invocations
and remap those that are supported to its own versions. These internal
versions determine if files on the command line are local or remote.
Remote files are processed transparently using a persistent SSH
connection to the associated host(s). Output associated with the local
and remote files is then multiplexed together into the standard unified
format associated with the original command. To the user, it appears as
if all files reside on a local file system even though they may span
multiple files systems on multiple hosts.
To install, copy "basshfs" to a directory in $PATH and "basshfs.1" to
a directory in $MANPATH. For usage details, see "basshfs.1" (in man
page format, viewable with "nroff -man").
Questions, comments, fixes, and/or enhancements welcome.
--Paul Kolano <paul.kolano@nasa.gov>

1551
basshfs 100755

File diff suppressed because one or more lines are too long

308
basshfs.1 100644
View File

@ -0,0 +1,308 @@
.TH "basshfs" "1" "06 Sep 2019" "" ""
./"################################################################
.SH "NAME"
./"################################################################
\fBbasshfs\fP \(em Bash-Accessible SSH File System
./"################################################################
.SH "SYNOPSIS"
./"################################################################
.nf
Mount: \fBeval `basshfs [USER@]HOST:[DIR] MOUNTPOINT [OPTIONS]`\fP
Unmount: \fBeval `basshfs -u MOUNTPOINT`\fP
List: \fBbasshfs -l\fP
.fi
.PP
Mount and unmount remote file systems as if they were local.
./"################################################################
.SH "DESCRIPTION"
./"################################################################
Working with remote systems over SSH is common in HPC environments
where the size of data sets makes them nontrivial to relocate. To run
arbitrary commands on that data, a full SSH session is required. There
are cases, however, when the user may wish to perform simpler operations
such as checking file existence and size, viewing differences between
configuration files, creating directories, etc. that can be achieved
with more limited access. Juggling multiple sessions to multiple hosts
may be inconvenient for such simple tasks. BASSHFS is a tool that
allows users to perform such tasks within a single terminal on a single
host by transparently carrying out remote operations as needed to
present remote files as if they are locally mounted when using the bash
shell.
.PP
BASSHFS is similar to the existing SSHFS utility except it is does not
require FUSE kernel support. Instead, BASSHFS uses the aliasing and
function mechanisms of the bash shell to intercept program invocations
and remap those that are supported to its own versions. These internal
versions determine if files on the command line are local or remote.
Remote files are processed transparently using a persistent SSH
connection to the associated host(s). Output associated with the local
and remote files is then multiplexed together into the standard unified
format associated with the original command. To the user, it appears as
if all files reside on a local file system even though they may span
multiple files systems on multiple hosts.
./"################################################################
.SH "REQUIREMENTS"
./"################################################################
BASSHFS functionality is only supported within the Bash shell and
requires SSH and Perl version 5.8.5 or above. It also requires the
standard Unix utilities cat, column, false, sort, and true and has been
tested successfully on Linux, OS X, and Windows under Cygwin. Note that
users of Windows under Cygwin may need to install the coreutils and
util-linux packages to obtain these utilities.
./"################################################################
.SH "USAGE"
./"################################################################
.IP "Mount file system"
Before a remote file system can be accessed, it must be "mounted"
locally. Note that this is not a true file system mount as with SSHFS,
but a virtual mount that is only known to certain commands within the
Bash shell currently running. The mount point must not correspond to
any existing file or directory. In order for the corresponding aliases
and functions to be imported into the existing Bash environment, the
command must be executed in an eval statement.
.PP
.RS
.RS
.nf
\fBeval `basshfs [USER@]HOST:[DIR] MOUNTPOINT [OPTIONS]`\fP
.fi
.RE
.RE
.IP
For example, to mount /home on host1 as /mnt/home1:
.PP
.RS
.RS
.nf
\fBeval `basshfs host1:/home1 /mnt/home1`\fP
.fi
.RE
.RE
.IP
By default, BASSHFS will access remote hosts using the bare "ssh"
command. If additional options are needed, the -s option may be used.
For example, if host1 can only be accessed by first hopping through
bastion1, the corresponding mount command would be:
.PP
.RS
.RS
.nf
\fBeval `basshfs -s "ssh bastion1 ssh" host1:/home1 /mnt/home1`\fP
.fi
.RE
.RE
.IP
For use within a non-interactive bash shell, the script must set the
expand_aliases option. For example:
.PP
.RS
.RS
.nf
#!/bin/bash
shopt -s expand_aliases
eval `basshfs host1:/home1 /mnt/home1`
.fi
.RE
.RE
.IP
The use of BASSHFS overrides previously defined aliases for supported
commands. For example, if "ls" was previously aliases to "ls
--color=always", once a file system is mounted, ls would no longer show
colorized output. If desired, default options for any supported command
can be specified using the -oCMD=OPTS option. For example, to set the
above option for ls, the following can be used:
.PP
.RS
.RS
.nf
\fBeval `basshfs -ols="--color=always" host1:/home1 /mnt/home1`\fP
.fi
.RE
.RE
.IP
Note that built-in commands support only a limited subset of the
available options found in their standard counterparts so options added
in this manner may only apply to local files and not to those that
reside on remote file systems.
.IP "Unmount file system"
To "unmount" a mounted file system and terminate any underlying
processes, the -u option is used. Similar to mount, in order for the
corresponding aliases and functions to be removed from the existing
Bash environment, the command must be executed in an eval statement.
.PP
.RS
.RS
.nf
\fBeval `basshfs -u MOUNTPOINT`\fP
.fi
.RE
.RE
.IP
For example, to unmount the previously mounted /mnt/home1:
.PP
.RS
.RS
.nf
\fBeval `basshfs -u /mnt/home1`\fP
.fi
.RE
.RE
.IP "List mounted file systems"
To see the list of mounted file systems, the -l option is used. Note
that it is not necessary to use an eval statement in this case.
.PP
.RS
.RS
.nf
\fBbasshfs -l\fP
.fi
.RE
.RE
./"################################################################
.SH "COMMANDS"
./"################################################################
Once a remote file system is mounted, a specific set of commands may
be run on locations within the mounted hierarchy as if the file system
were local. Tab completion is also supported normally. Currently
supported commands and their currently supported options are below.
Unsupported options will simply be ignored except where noted.
.IP "\fBcat\fP (no options)"
.IP "\fBcd\fP (no options)"
Note that when changing to remote directories, cd only changes
$PWD so to make changes visible, the working directory (i.e. \w in
bash) must be in your prompt. For example, the following prompt:
.PP
.RS
.RS
.nf
export PS1="\\h:\\w> "
.fi
.RE
.RE
.IP
would display the current host name followed by the current
working directory.
.IP "\fBchgrp\fP (no options)"
Groups may be specified either by number or by name. Names will be
resolved on the remote host.
.IP "\fBchmod\fP (no options)"
Modes must be specified numerically (e.g. 0700). Symbolic modes, such
as a+rX, are not currently supported.
.IP "\fBchown\fP (no options)"
Users and groups may be specified either by number or by name. Names
will be resolved on the remote host.
.IP "\fBcmp\fP (all options)"
.IP "\fBcp\fP [-r]"
Note that copies between two remote hosts transfer files to the local
host first since BASSHFS does not allow third party transfers. Thus,
very large file transfers between remote systems should be achieved
using an alternate approach.
.IP "\fBdf\fP [-i]"
Note that 1024-byte blocks are used.
.IP "\fBdiff\fP (all options)"
.IP "\fBdu\fP [-a] [-b] [-s]"
Note that 1024-byte blocks are used.
.IP "\fBfile\fP (all options)"
.IP "\fBgrep\fP (all options)"
.IP "\fBhead\fP [-number]"
Note that head does not support the form "-n number", thus, for
example, to display the first 5 lines of a file, use "-5" and not "-n
5".
.IP "\fBless\fP (all options)"
.IP "\fBln\fP [-s]"
Note that hard links are not supported.
.IP "\fBls\fP [-1] [-d] [-l]"
For efficiency purposes, ls behaves slightly differently for remote
commands than for local. In particular "ls -l" will not show links by
default and will show what is actually linked instead of the link
itself. Link details can be obtained using the "-d" option (e.g. ls -ld
*).
.IP
Also for efficiency, ls processes remote files before local files, so
output ordering may be changed when remote and local files are
interleaved on the ls command line. For example, "ls /foo /mnt/host1
/bar" would show /mnt/host1 first, then /foo, then /bar.
.IP "\fBmkdir\fP (no options)"
.IP "\fBmore\fP (all options)"
.IP "\fBmount\fP (all options)"
.IP "\fBmv\fP (no options)"
.IP "\fBpwd\fP (no options)"
.IP "\fBrm\fP [-r]"
.IP "\fBrmdir\fP (no options)"
.IP "\fBtail\fP [-number]"
Note that tail does not support the form "-n number", thus, for
example, to display the last 5 lines of a file, use "-5" and not "-n 5".
.IP "\fBtee\fP [-a]"
.IP "\fBtest\fP [-b] [-c] [-d] [-e] [-f] [-g] [-h] [-k] [-L] [-p] [-r] [-s] [-S] [-u] [-w]"
Note that compound and string tests are not supported. Compound and
string tests can be achieved using multiple test commands separated by
shell compound operators. For example,
.PP
.RS
.RS
.nf
test -f /mnt/host1/foo -a "abc" != "123"
.fi
.RE
.RE
.IP
would become:
.PP
.RS
.RS
.nf
test -f /mnt/host1/foo && test "abc" != "123"
.fi
.RE
.RE
.IP "\fBtouch\fP (no options)
.IP "\fBwc\fP (all options)
./"################################################################
.SH "CAVEATS"
./"################################################################
In general, BASSHFS works for the most common usage scenarios with
some caveats. In particular:
.IP -
"Whole file" commands (i.e. commands that must process the entire
file), including cat, cmp, diff, grep, wc (and currently more/less due
to implementation) retrieve files first before processing for
efficiency. Thus, these commands should not be executed on very large
files.
.IP -
There is a conflict between commands that take piped input and the
custom globbing utilized by BASSHFS, thus these commands have portions
of globbing support disabled. These commands are grep, head, less,
more, tail, tee, and wc. In these cases, globbing will work for
absolute prefixes, but not relative. For example, "grep foo
/mnt/host1/tmp/*" will work, but "cd /mnt/host1/tmp; grep foo *" will
not.
.IP -
Redirection to/from remote files doesn't work. The same effect can be
achieved using cat and tee (e.g. "grep localhost </mnt/host1/etc/hosts
>a" would become "cat /mnt/host1/etc/hosts |grep localhost | tee -a
>/dev/null"). Redirection still works normally for local files.
.IP -
The first time a command is run involving a particular host, a SFTP
connection is created to that host. When running "ps", it may appear as
if a zombie client process is running.
.IP -
Commands may hang the first time after switching networks (e.g. with
a laptop). If this happens, hit Control-c and it will work the next
time.
.IP -
The ls command does not show BASSHFS mount points.
.IP -
The wc command shows the names of the local copies of the file
arguments instead of the original names.
.IP -
Various commands and completion may expand relative paths to absolute
paths.
./"################################################################
.SH "AUTHOR"
./"################################################################
BASSHFS was written by Paul Kolano.
./"################################################################
.SH "SEE ALSO"
./"################################################################
bash(1), sshfs(1)