Newer
Older
#!/usr/bin/env python3
# Copyright (c) 2020 Open Tech Strategies, LLC
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published
# by the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
# If you did not receive a copy of the GNU Affero General Public License
# along with this program, see <http://www.gnu.org/licenses/>.
__doc__ = """\
Collaboration porcelain around the password-store utility 'pass'.
'opass' allows multiple GPG-enabled people to manage semi-shared
secrets in a tree. Different subtrees can be encrypted for different
people. We use 'opass' internally at https://OpenTechStrategies.com/
and it has some OTS idiosyncracies. Our corporate tree is kept in
Subversion, so opass uses git-svn as a bridge, since 'opass' is based
on 'pass' (https://www.passwordstore.org/), which uses Git. Thus
'opass' requires https://github.com/opentechstrategies/pass-git-svn to
be installed. You will also see some references to \"OTS_DIR\", an
environment variable that points to the root of the user's local SVN
checked-out tree. 'opass' should still work without OTS_DIR, but note
If you use the Bash shell, you might try 'opass-bash-completion' too (see
https://code.librehq.com/ots/ots-tools/-/blob/main/opass-bash-completion).
If you have any questions, ask at https://chat.opentechstrategies.com/.
We welcome patches to improve this program, including changes to make
it less OTS-specific.
Usage: opass COMMAND [-o|--offline] [SERVICE_NAME] [NEW_SERVICE_NAME]
COMMAND is one of the following:
- list [PREFIX] # 'ls' works too
- update # 'up' works too, as does 'sync'
- edit NAME_OF_SERVICE # use this to edit existing or to add new
- show NAME_OF_SERVICE # 'get' or 'fetch' works too
- mv NAME_OF_SERVICE NEW_NAME_OF_SERVICE
- rm NAME_OF_SERVICE
- ln NAME_OF_EXISTING_SERVICE NAME_OF_NEW_LINK # 'ln' for 'link'
- add-key USERNAME_OR_KEY_ID NAME_OF_DIRECTORY
- del-key USERNAME_OR_KEY_ID NAME_OF_DIRECTORY # 'rm-key' works too
- list-keys [NAME_OF_DIRECTORY] # 'ls-keys' works too
- push # equivalent to 'pass git-svn dcommit'
- otp NAME_OF_SERVICE # show an OTP authn code; see section
# "Obtaining OTP (TOTP) codes" below
Wherever you see \"USERNAME_OR_KEY_ID\" above, the \"USERNAME\" refers
to an opass user as listed in a \"USERS\" file at the top of the
.password-store/ tree. The format of the USERS file is simple: each
line is a username, followed by a colon and a space, followed by a
standard 40-character GPG fingerprint. You can add new usernames by
editing ${PASSWORD_STORE_DIR}/USERS directly. Note that there is
no requirement to have a USERS file at all; you can always just use
key IDs directly. The USERS file just makes adding, removing, and
listing keys a little bit more human-friendly. If you have a USERS
file, then running the command 'opass list-keys' without any arguments
will show all the users in the file and their keys.
The 'ln' command creates a new service file that forwards to an
existing service FOO by having content that just says "See FOO." It
is analogous to a Unix symlink, but a human has to interpret it.
If you run the 'list' command with no argument, it will list all
files in the password store. You can filter the list by passing a
PREFIX. Note that this is purely a string-prefix match and does not
take into account path components. That is, 'opass list some/path'
will match 'some/path/' and 'some/path-to-another/file', but not
'some/other/path' or 'secret/some/path', and 'opass list another/path/'
will not match 'another/path'.
If the "-o" ("--offline") option is passed, then perform only
local operations and do not attempt to sync with the OTS repository.
(Note that this option cannot be used with "update" or "push".)
When operating online, you shouldn't normally need to use the 'push'
command; opass will automatically do pushes for you. But if you do
stuff offline, then when you get back online you should run 'push'.
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
Obtaining OTP (TOTP) codes (for 2FA/MFA authn):
===============================================
The 'otp' subcommand relies on the otp extension being installed for
the underlying 'pass' program -- i.e., 'pass otp NAME_OF_SERVICE'
needs to work. On Debian-based distributions, you can install that
extention by doing 'apt-get install pass-otp'; you can also install it
manually from https://github.com/tadfisher/pass-otp/.
Then the NAME_OF_SERVICE file needs a line like this somewhere in it
(we think it will work if that line is anywhere in the file, but Your
Mileage May Vary -- we've tested it most at the bottom):
otpauth://totp/OAuthProvider:jrandom?secret=B2008251645EB2EE03AAD745E3288EEB&issuer=SomeServiceProvider
In that string, the parts that matter are "otpauth://totp/" and
"secret=foo". The other parts might have values different from
"OAuthProvider:jrandom" and "&issuer=SomeServiceProvider", of course,
depending on where you got the particular TOTP secret.
Once you have that in the NAME_OF_SERVICE file, you can generate a
TOTP code with a command like this:
$ opass otp NAME_OF_SERVICE
218653
$
class NoAuthzFound(Exception):
"No public keys were found at all for the password store."
class NoSuchDirectory(Exception):
"There is no encrypted (.gpg) file for this directory path."
class PathIsNotDirectory(Exception):
"The provided service path is a leaf node, not a directory path."
"""Return the root of the password store as an absolute path,
with no trailing slash."""
path = os.environ.get("PASSWORD_STORE_DIR")
if path is not None:
# It shouldn't have a trailing slash, but we don't know how
# the user has set the environment variable, so make sure.
return os.path.normpath(path)
else:
return os.path.expanduser("~/.password-store")
setup_message = """
If you got an error about the password store not being a git-svn
repository, or about being unable to open ~/.password-store/USERS,
then you need to initialize your ~/.password-store directory:
$ cd ${HOME}
$ mkdir .password-store
$ cd .password-store
$ pass git-svn clone \\
https://svn.opentechstrategies.com/repos/ots/trunk/.password-store
## (Note that this step will take a while.) ##
$ cd ..
If you got an error about how git-svn is not in the password store,
then you need to install the 'pass-git-svn' extension to 'pass':
https://code.librehq.com/ots/pass-git-svn
# One dictionary maps usernames to long-form GPG fingerprints, the
# other does the reverse. Note that at certain places in the code
# we assume that there is no username that is also someone's long-form
# GPG fingerprint. So if one of these dictionaries maps X->Y, then we
# assume the other maps Y->X and does *not* map X->anything.
"""Print MSG to stderr, prefixed with "ERROR: ", and exit w/ error.
sys.stderr.write("ERROR: " + str(msg).strip() + "\n")
def set_up_usermap():
"""Set up the globals 'users_to_fprints' and 'fprints_to_users'."""
global users_to_fprints
global fprints_to_users
try:
with open(ufile) as f:
for line in f:
line = line.strip()
if line:
username, fingerprint = line.split(":")
fingerprint = fingerprint.strip()
users_to_fprints[username] = fingerprint
fprints_to_users[fingerprint] = username
except FileNotFoundError as err:
global setup_message
err_exit("Unable to open '%s'\n" % ufile + setup_message)
err_exit("Something unexpected went wrong:\n '%s'\n" % err)
def canonicalize_as_directory(path):
"""Canonicalize PATH as a directory path within the password store.
If PATH is None or "/", then return "" (the root of the password
store tree). Otherwise, return PATH with any trailing slash stripped.
if path is None or path == "/":
return ""
else:
return os.path.normpath(path)
def get_directory_path_authz(directory):
"""Return the list of pubkeys (strings) that have access to DIRECTORY.
DIRECTORY is an intermediate service path within the password store
(e.g., "infra/hosting" but not "infra/hosting/linode"). DIRECTORY
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
must already be in canonical form; see canonicalize_as_directory().
Raise a NoSuchDirectory exception if DIRECTORY does not exist.
Raise a PathIsNotDirectory exception if DIRECTORY is only a
leaf node and is not a directory path.
Raise a NoAuthzFound exception if no pubkeys could be found at all.
Print a warning on stderr if DIRECTORY is a leaf service path.
(Printing a warning on stderr makes this function unsuitable for
usage in a library. I didn't feel like complexifying the return
signature, but if we ever librarize this code, we'll have to lose
the stderr output and do that instead.)
"""
# Convert to a filesystem path.
root = password_store_root()
directory = os.path.join(root, directory)
# Check for leaf vs directory situations.
is_leaf = os.path.isfile(directory + ".gpg")
is_directory = os.path.isdir(directory)
if is_directory and is_leaf:
# There's a weirdness in the password-store system: a given
# path can be both a directory and a leaf service path (i.e.,
# a file). While that's not possible in a normal filesystem,
# it's possible here because the actual encrypted files in
# which passwords are stored have ".gpg" extensions that are
# never shown -- so, under the hood, their name does differ
# from a directory that has the "same name", but what a user
# sees is two objects of different type that have exactly the
# same path. The upstream 'pass' program doesn't warn about
# this, but 'opass' does.
sys.stderr.write(
"WARNING: '%s'\n"
" is both a directory path and a leaf service path.\n"
" This is likely to cause confusion.\n"
elif is_leaf: # is only a leaf
raise NoSuchDirectory(
"ERROR: '%s'\n"
" is a leaf service path, not a directory path.\n"
" Access control is done at the directory level,\n"
raise NoSuchDirectory("ERROR:\n '%s' is not a directory path." % directory)
# Okay, now we can start looking for .gpg-id access control files.
# Because a given key list overrides any higher-level key list,
# the most recently-seen '.gpg-id' file is the only thing that
# matters. So, walk up until the first '.gpg-id' file we find.
while len(directory) >= len(root):
this_gpg_id_file = os.path.join(directory, ".gpg-id")
if os.path.isfile(this_gpg_id_file):
with open(this_gpg_id_file, "r") as f:
return [x.rstrip() for x in f.readlines()]
else:
directory, ignored = os.path.split(directory)
# In practice we should never get here, because there should be at
# least a top-level '.gpg-id' file in the password store.
raise NoAuthzFound("No public keys in the password store for '%s'." % directory)
def check_extension(ext: str) -> None:
"""Check extension setup and existence, exit if bad.
EXT is the name of an extension (e.g. git-svn or otp)
* Check env vars enable extensions and specify an extension dir.
* Check that EXT is installed and executable
If any check fails, complain to user and exit"""
# Check for the value that 'pass' itself requires.
if os.environ.get("PASSWORD_STORE_ENABLE_EXTENSIONS", "") != "true":
err_exit("Environment variable PASSWORD_STORE_ENABLE_EXTENSIONS\n"
"must be set to \"true\".")
# Mirror the default that 'pass' itself falls back to.
psed = os.environ.get("PASSWORD_STORE_EXTENSIONS_DIR",
os.path.join(password_store_root(), ".extensions"))
ext_path = os.path.join(psed, f"{ext}.bash")
if not os.path.exists(ext_path):
msg = f"Cannot find {ext_path}. Please install\n{ext} and tell pass where to find it."
if ext == "git-svn":
msg += " For git-svn, see 'pass-git-svn':\n\n https://code.librehq.com/ots/pass-git-svn"
elif ext == "otp":
msg += " Debian repos have otp:\n\n apt install pass-otp"
if not os.access(ext_path, os.X_OK):
err_exit(f"{ext_path} is not executable. Please make it executable.\n")
def main():
# These will come from arguments.
cmd = None
service = None
new_service = None # used in case of 'mv' command
pub_key = None
that happens to be useful in a few message strings.
If USERNAME is None, return the empty string."""
if username is not None:
return " (" + username + ")"
else:
return ""
# Assume online by default; false when -o / --offline option
#
# If anyone knows a reliable way to autodetect whether one's box
# is on the Net or not, let me know. I mean, the script could
# ping some well-known site, but that would be both slow and a
# violation of the privacy principle that a program generally
# shouldn't make noises that are audible beyond the user except
(opts, args) = getopt.getopt(
sys.argv[1:],
"hov",
[
"help",
"offline",
"verbose",
],
)
if len(args) == 0:
cmd = "list"
elif len(args) == 1:
cmd = args[0]
elif len(args) == 2:
cmd = args[0]
service = args[1]
elif len(args) == 3:
cmd = args[0]
if cmd.find("-key") != -1:
# If somone passes a username, convert it right away.
pub_key = users_to_fprints.get(args[1], args[1])
service = args[2]
else:
service = args[1]
new_service = args[2]
else:
err_exit(f"Unexpected number of arguments\n\n{__doc__}")
err_exit("Cannot find 'pass' program in PATH.")
# We don't use the default list output of 'pass', because
# that's a tree-style display (using the 'tree' utility)
# that's not actually very useful for our purposes. We want
# listings presented in a format amenable to copying and
# pasting arguments to a subsequent 'opass' command.
# So we have our own function to print all the paths that
# could be valid arguments to opass. That is, the relative
# path (under ~/.password-store) of every file ending in
# ".gpg", but with the ".gpg" suffix stripped off.
#
# If it turns out that some people really like the tree-style
# output, we can easily add it back as an option, controlled
# by a flag, a subcommand, an environment variable, whatever.
#
# If PREFIX is provided, this will only print paths that start
# with that PREFIX. Note that this is purely a string-prefix
# match and does not take into account path components.
pass_root = password_store_root()
for dirpath, _, filenames in os.walk(pass_root):
# subpath within password_store_root
for filename in filenames:
if filename.endswith(".gpg"):
# Remove leading slash and trailing ".gpg"
if prefix and not path.startswith(prefix):
check_extension("git-svn")
print("About to execute the following 'pass git-svn' command:")
completed = subprocess.run(
args,
capture_output=True,
shell=False,
encoding="UTF-8",
text=True,
check=True,
)
# We filter some lines out because they're common and
# carry no useful information. Everything else we print,
# but always to stderr, whether it came from stdout or
# stderr. That's because the output here isn't the main
# purpose of opass, but is just a side effect of the
# surrounding toolchain, and having noise on stdout was
# affecting scripts that use opass to automatically fetch
# secrets for use with, e.g., ansible.
for line in completed.stdout.splitlines(keepends=True):
if line.find("Current branch main is up to date.") != -1:
pass
else:
sys.stderr.write("%s" % line)
for line in completed.stderr.splitlines(keepends=True):
sys.stderr.write("%s" % line)
except subprocess.CalledProcessError as e:
err_exit(e.stderr + "\n" + setup_message)
def do_pass_thing(things, sync=False, content=None):
"""THINGS is an argument tuple to 'pass', starting with subcommand.
For example: ['edit', service_name,].
If SYNC, then sync with upstream before and after.
If CONTENT is not None, then send it to 'pass' on stdin (in
which case the command in THINGS[0] must be 'insert')."""
if content is not None and things[0] != "insert":
err_exit("Can only send content if " "'pass' subcommand is 'insert'.\n")
if sync and online_p:
do_git_svn_thing("rebase")
maybe_changed = True
subprocess.run(
args, input=content, shell=False, encoding="UTF-8", check=True
)
except subprocess.CalledProcessError as err:
if (err.returncode == 1) and (err.output is None) and (err.stderr is None):
# If we got an error but all the above conditions hold,
# then it just means the user made no changes to the
# file, so there's no need for us to dcommit later.
#
# Alternatively, we might be here because the user tried to do a
# pass thing to a file that doesn't exist.
#
# Either way, time to bail out. And we'll indicate an error
# because either something went wrong or nothing left to do.
do_git_svn_thing("dcommit")
if cmd == "ls" or cmd == "list":
if online_p:
do_git_svn_thing("rebase")
elif cmd == "up" or cmd == "update" or cmd == "sync":
if not online_p:
err_exit("Cannot use -o / --offline " 'with "update" command')
do_git_svn_thing("rebase")
elif cmd == "add-key":
# I did my Ph.D. thesis on key management in opass. But you
# probably don't have time to read my whole thesis, nor the
# bestselling popular-audience book based on it, and you
# probably didn't see the movie. So let me summarize things
# for you:
#
# The password store's inheritance behavior is supremely weird
# from a user perspective, although perfectly understandable
# from an implementation perspective.
#
# Imagine we're working with this subtree:
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
#
# ./.gpg-id
# ./test.gpg
# ./test/
# ./test/subdir-1.gpg
# ./test/subdir-1/
# ./test/subdir-1/subdir-2.gpg
# ./test/subdir-1/subdir-2/.gpg-id
# ./test/subdir-1/subdir-2/subdir-3.gpg
#
# (The 'subdir-N.gpg' files are just regular encrypted files;
# despite their names, they aren't directory-ish in any way.
# I just gave them those names in order to demonstrate
# something in the example below.)
#
# Remember, the two '.gpg-id' files are special -- they hold
# lists of public keys, one per line. Assume they contain
# these keys:
#
# ./.gpg-id ==> only keys PUB_A and PUB_B
# ./test/subdir-1/subdir-2/.gpg-id ==> only key PUB_C
#
# Let's say we got to this state by starting out with *just*
# the top-level ./.gpg-id. Originally, that meant the owners
# of keys PUB_A and PUB_B could see everything in the entire
# tree, and no one else could see anything.
# But then we ran 'pass init -p test/subdir-1/subdir-2 PUB_C'.
# Afterwards, we get the full tree state shown above with two
# '.gpg-id' files, and the following are all true:
#
# - PUB_A and PUB_B can read everything *except* 'subdir-3.gpg'
# - PUB_C can *not* read subdir-2.gpg
# - PUB_C *can* read subdir-3.gpg (while PUB_A and PUB_B cannot)
#
# In other words, each '.gpg-id' file represents a fully
# controlling authz cone for the subtree underneath it, with
# no inheritance from ancestor '.gpg-id' files. However, a
# '.gpg-id' file's cone does not apply to foo.gpg files that
# are in the same directory as the '.gpg-id' file in question,
# and this doesn't change just because one of those foo.gpg
# files coincidentally has the same basename (sans extension)
# as a sibling subdirectory.
# So what should the behavior of 'opass' be?
#
# Several things:
#
# - Since authz control in 'pass' is only per-directory /
# per sub-tree, never per-file, we can only allow authz
# ops on directories, never on leaf services themselves.
# If the user tries to do an authz op on a leaf service,
# they should get an error.
#
# - Keep the UI simple. In practice, we're always granting
# one person access to one subtree at a time, so the UI can
# reflect that. It doesn't need to support adding/deleting
# multiple keys in a single command or anything like that.
#
# - The UI we want is 'opass add-key PUB_KEY DIRECTORY'
# and 'opass del-key PUB_KEY DIRECTORY'. No one else's
# access should be affected when one of these operations is
# performed. Whichever operation it is, it's only about
# PUB_KEY, not about other keys.
# - When a target directory 'foo' also has a sibling
# encrypted file 'foo.gpg', we should issue a warning that
# the authz for the 'foo' service (that is, for 'foo.gpg')
# will not be affected by the operation, that this is
# likely to cause confusion, and that renaming the leaf
# service would eliminate that confusion.
directory = canonicalize_as_directory(service)
current_keys = get_directory_path_authz(directory)
if pub_key in current_keys:
sys.stderr.write(
"WARNING: public key '%s'%s already "
"authorized for '%s'\n"
% (pub_key, inline_username(fprints_to_users.get(pub_key)), directory)
)
sys.stderr.write(
" (Use 'list-keys %s' to see all keys "
"authorized for that directory.)\n" % directory
)
do_pass_thing(
[
"init",
"-p",
directory,
]
+ current_keys
+ [
pub_key,
],
sync=True,
)
elif cmd == "del-key" or cmd == "rm-key":
directory = canonicalize_as_directory(service)
current_keys = get_directory_path_authz(directory)
if pub_key not in current_keys:
sys.stderr.write(
"WARNING: public key '%s'%s is not currently "
"authorized for '%s'\n"
% (pub_key, inline_username(fprints_to_users.get(pub_key)), directory)
)
sys.stderr.write(
" (Use 'list-keys %s' to see all keys "
"authorized for that directory.)\n" % directory
)
else:
# Using current_keys.remove() wouldn't be good enough,
# because the key might occur more than once, and we need
# to make absolutely sure we have removed it completely.
# Also, we may want to do a fuzzier matching test than
# just equality eventually. So for all these reasons,
# filtering the entire current key set is the way to go.
new_keys = [x for x in current_keys if x != pub_key]
do_pass_thing(
[
"init",
"-p",
directory,
]
+ new_keys,
sync=True,
)
elif cmd == "list-keys" or cmd == "ls-keys":
if service is not None:
directory = canonicalize_as_directory(service)
print("Public keys authorized for '%s':" % directory)
for key in get_directory_path_authz(directory):
print(" - %s%s" % (key, inline_username(fprints_to_users.get(key))))
print(f"All keys registered in {password_store_root()}/USERS:")
for key, val in users_to_fprints.items():
print(" - %s (%s)" % (val, key))
elif cmd == "get" or cmd == "fetch" or cmd == "show":
if online_p:
do_git_svn_thing("rebase")
# NOTE: It would be nice to bring the information up in a buffer
# in emacsclient, but emacsclient doesn't take input from stdin,
# so the only way to do it would be to write the information to a
# tmp file and invoke emacsclient on the tmp file. But writing
# passwords and such into tmp files seems like something that many
# users would prefer not happen, so for now, we just do whatever
# 'pass show' does, which is print it on stdout.
# We do not pass the -c option to put the information in the
# clipboard, for two reasons. One, many of our pass files are
# multiline affairs containing descriptive data as well as the
# actual authn creds, so putting all that in the clipboard
# wouldn't be very useful. Two, doing so would violate the
# Principle of Least Surprise -- the last thing the user needs is
# to, say, go to Google or some other web form and accidentally
# paste in a bunch of OTS secrets.
if service is not None:
do_pass_thing(
[
"show",
service,
],
sync=False,
)
do_pass_thing(
[
"edit",
service,
],
sync=True,
)
do_pass_thing(
[
"mv",
service,
new_service,
],
sync=True,
)
do_pass_thing(
[
"insert",
"--echo",
new_service,
],
sync=True,
content="See '%s'." % service,
)
do_pass_thing(
[
"rm",
service,
],
sync=True,
)
do_pass_thing(
[
"otp",
service,
],
sync=False,
)
err_exit("Cannot use -o / --offline " 'with "push" command')
do_git_svn_thing("dcommit")
elif cmd == "help":
print("%s" % __doc__)
sys.exit(0)
else:
err_exit("Unknown command '%s'" % cmd)