NAME
git-annex addurl - add urls to annex
SYNOPSIS
git annex addurl [url ...]
DESCRIPTION
Downloads each url to its own file, which is added to the annex.
When youtube-dl
is installed, it can be used to check for a video
embedded in a web page at the url, and that is added to the annex instead.
(However, this is disabled by default as it can be a security risk.
See the documentation of annex.security.allowed-http-addresses
in git-annex(1) for details.)
Urls to torrent files (including magnet links) will cause the content of
the torrent to be downloaded, using aria2c
.
Normally the filename is based on the full url, so will look like "www.example.com_dir_subdir_bigfile". In some cases, addurl is able to come up with a better filename based on other information. Options can also be used to get better filenames.
OPTIONS
--fast
Avoid immediately downloading the url. The url is still checked (via HEAD) to verify that it exists, and to get its size if possible.
--relaxed
Don't immediately download the url, and avoid storing the size of the url's content. This makes git-annex accept whatever content is there at a future point.
This is the fastest option, but it still has to access the network to check if the url contains embedded media. When adding large numbers of urls, using
--relaxed --raw
is much faster.--raw
Prevent special handling of urls by youtube-dl, bittorrent, and other special remotes. This will for example, make addurl download the .torrent file and not the contents it points to.
--file=name
Use with a filename that does not yet exist to add a new file with the specified name and the content downloaded from the url.
If the file already exists, addurl will record that it can be downloaded from the specified url(s).
--pathdepth=N
Rather than basing the filename on the whole url, this causes a path to be constructed, starting at the specified depth within the path of the url.
For example, adding the url http://www.example.com/dir/subdir/bigfile with
--pathdepth=1
will use "dir/subdir/bigfile", while--pathdepth=3
will use "bigfile".It can also be negative;
--pathdepth=-2
will use the last two parts of the url.--prefix=foo
--suffix=bar
Use to adjust the filenames that are created by addurl. For example,
--suffix=.mp3
can be used to add an extension to the file.--jobs=N
-JN
Enables parallel downloads when multiple urls are being added. For example:
-J4
--batch
Enables batch mode, in which lines containing urls to add are read from stdin.
--with-files
When batch mode is enabled, makes it parse lines of the form: "$url $file"
That adds the specified url to the specified file, downloading its content if the file does not yet exist; the same as
git annex addurl $url --file $file
--json
Enable JSON output. This is intended to be parsed by programs that use git-annex. Each line of output is a JSON object.
--json-progress
Include progress objects in JSON output.
--json-error-messages
Messages that would normally be output to standard error are included in the json instead.
CAVEATS
If annex.largefiles is configured, and does not match a file, git annex
addurl
will add the non-large file directly to the git repository,
instead of to the annex. However, this is not done when --fast or --relaxed
is used.
SEE ALSO
git-annex(1)
AUTHOR
Joey Hess id@joeyh.name
Warning: Automatically converted into a man page by mdwn2man. Edit with care.
I have been trying to figure out how to use addurl to get this video. I have this in my mscourtstuff annex as a large binary, but I would really like to use the web as a remote for this.
Hughes v Hosemann 2010-CA-01949-SCT-43112001.mp4 youtube-dl --referer 'http://judicial.mc.edu/case.php?id=24206' http://player.vimeo.com/video/43112001
There's not currently a way to do per-file youtube-dl options. The difficulty is that we don't know what youtube-dl options might be unsafe, and which such a feature could make eg
git annex get
use when run by a different user.I feel that this needs some support in youtube-dl to avoid git-annex needing to know about all its safe options. Especially since which options are available, or safe, could vary between versions of youtube-dl.