Use full URL as saved file name with wget

wget redirect output to file
wget command
wget save file to directory
wget local file
wget rename
wget download all files
wget man page ubuntu
wget with multiple files

I'm using wget in terminal to download a large list of images.

example — $ wget -i images.txt

I have all the image URLS in the images.txt file.

However, the image urls tend to be like example.com/unqiueNumber/images/main_250.jpg

which means that all the images come out named main_250.jpg

What I really need is the images to be saved with the entire URLs for the images for each one, so that the 'unique number' is part of the filenames.

Any suggestions?


Saving a file downloaded with wget with a different name, This would save the icon file with the filename linux-bsd.gif into the current directory. part of the webpage URL) then those will also be included in the filename. It is possible to save the downloaded file with a different filename using wget,  By default when you download a file with wget, the file will be written to the current directory, with the same name as the filename in the URL. For example, if you were to download the little Tux penguin and BSD demon icon which is on this page, you would use wget like so:


With GNU Parallel you can do:

cat images.txt | parallel wget -O '{= s:/:-:g; =}' {}

How to wget a file with correct name when redirected?, The documents will not be written to the appropriate files, but all will be (Use ./- to print to a file literally named -. ) So, wget -O somefile.extension -O to get that name from the content-disposition header rather than the URL, and -L follows redirects if needed. to save the file using the last file name the server gives you. I am downloading a file using the wget command. But when it downloads to my local machine, I want it to be saved as a different filename. For example: I am downloading a file from www.examplesite.


I have a not so elegant solution, that may not work everywhere.

You probably know that if your URL ends in a query, wget will use that query in the filename. e.g. if you have http://domain/page?q=blabla, you will get a file called page?q=blabla after download. Usually, this is annoying, but you can turn it to your advantage.

Suppose, you wanted to download some index.html pages, and wanted to keep track of their origin, as well as, avoid ending up with index.html, index.html.1, index.html.2, etc. in your download folder. Your input file urls.txt may look something like the following:

https://google.com/
https://bing.com/
https://duckduckgo.com/

If you call wget -i urls.txt you end up with those numbered index.html files. But if you "doctor" your urls with a fake query, you get useful file names.

Write a script that appends each url as a query to itself, e.g.

https://google.com/?url=https://google.com/
https://bing.com/?url=https://bing.com/
https://duckduckgo.com/?url=https://duckduck.com/

Looks cheesy, right? But if you now execute wget -i urls.txt, you get the following files:

index.html?url=https:%2F%2Fbing.com%2F
index.html?url=https:%2F%2Fduckduck.com%2F
index.html?url=https:%2F%2Fgoogle.com%2F

instead of non-descript numbered index.htmls. Sure, they look ugly, but you can clean up the filenames, and voilà! Each file will have its origin as its name.

The approach probably has some limitations, e.g. if the site you are downloading from actually executes the query and parses the parameters, etc.

Otherwise, you'll have to solve the file name/source url problem outside of wget, either with a bash script or in other programming languages.

Download Files with Wget on the Linux Shell, Please note that all the examples and instructions mentioned in this article have been Here's how you can use Wget to download a file: wget [URL] By default​, wget saves the file with the same name it carries on the Web. options - The Wget options; url - URL of the file or directory you want to download or synchronize.; How to Download a File with Wget #. In it’s simplest form when used without any option, wget will download the resource specified in the [url] to the current directory.


Downloading files with wget, wget [options] url If the file was called filename-4.0.1.zip and you wanted to save it directly to filename.zip you would use a command like this: To use this, all the links in the file must be full links, if they are relative links you  If you want to download a large file and close your connection to the server you can use the command: wget -b url Downloading Multiple Files. If you want to download multiple files you can create a text file with the list of target files. Each filename should be on its own line. You would then run the command: wget -i filename.txt. You can also


Linux wget store with different file name, By default, downloaded file will be saved with the last name mentioned in the URL. To save file with a different name option O can be used. Syntax: wget -O  The wget command can be used to download files using the Linux and Windows command lines. wget can download entire websites and accompanying files.


How do I use wget with a list of URLs and their corresponding output , How do I use wget to download that list of URLs and save the returned data to the proper You can tell wget to use the name from the redirected URL (i.e. it could lead to overwriting an unpredictable file name in the current directory; Pass the --content-disposition option to make wget use this file name. The file is retrieved and saved to disk. We can use ls to check the file details. It has the same name as the file on the FTP server, and it is the same length, 403 bytes. ls -hl readme.txt. RELATED: How to Use the FTP Command on Linux. Sending Parameters to Remote Servers. Some remote servers will accept parameters in requests that are sent to