How can I download a web server’s directory and all subdirectories with one command?
15th January 2012
My answer to How can I download a web server’s directory and all subdirectories with one command? on Quora
Use wget (you can install it with apt-get install wget)
$ wget --recursive http://example.com
That will create a directory called example.com and put the mirrored downloaded files in the right sub-directories inside it.
If you just want to download a subdirectory, do this:
$ wget --recursive http://example.com/subdirectory --no-parent
The --no-parent option ensures wget won’t follow links up to parent directories of the one you want to download.
More recent articles
- Weeknotes: the Datasette Cloud API, a podcast appearance and more - 1st October 2023
- Things I've learned about building CLI tools in Python - 30th September 2023
- Talking Large Language Models with Rooftop Ruby - 29th September 2023
- Weeknotes: Embeddings, more embeddings and Datasette Cloud - 17th September 2023
- Build an image search engine with llm-clip, chat with models with llm chat - 12th September 2023
- LLM now provides tools for working with embeddings - 4th September 2023
- Datasette 1.0a4 and 1.0a5, plus weeknotes - 30th August 2023
- Making Large Language Models work for you - 27th August 2023
- Datasette Cloud, Datasette 1.0a3, llm-mlc and more - 16th August 2023
- How I make annotated presentations - 6th August 2023