10 Wget (Linux File Downloader) Command Examples in Linux

In this post we are going to review wget utility which retrieves files from World Wide Web (WWW) using widely used protocols like HTTP, HTTPS and FTP. Wget utility is freely available package and license is under GNU GPL License. This utility can be install any Unix-like Operating system including Windows and MAC OS. It’s a non-interactive command line tool. Main feature of Wget of it’s robustness. It’s designed in such way so that it works in slow or unstable network connections. Wget automatically start download where it was left off in case of network problem. Also downloads file recursively. It’ll keep trying until file has be retrieved completely.

10 Wget (Linux File Downloader) Command Examples in Linux

10 Linux Wget Command Examples

First, check whether wget utility is already installed or not in your Linux box, using following command.

# rpm -qa wget

wget-1.12-1.4.el6.i686

Please install it using YUM command in case wget is not installed already or you can also download binary package at http://ftp.gnu.org/gnu/wget/.

# yum -y install wget

The -y option used here, is to prevent confirmation prompt before installing any package. For more YUM command examples and options read the article on 20 YUM Command Examples for Linux Package Management.

1. Single file download

The command will download single file and stores in a current directory. It also shows download progress, size, date and time while downloading.

# wget http://ftp.gnu.org/gnu/wget/wget-1.5.3.tar.gz

--2012-10-02 11:28:30--  http://ftp.gnu.org/gnu/wget/wget-1.5.3.tar.gz
Resolving ftp.gnu.org... 208.118.235.20, 2001:4830:134:3::b
Connecting to ftp.gnu.org|208.118.235.20|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 446966 (436K) [application/x-gzip]
Saving to: wget-1.5.3.tar.gz
100%[===================================================================================>] 446,966     60.0K/s   in 7.4s
2012-10-02 11:28:38 (58.9 KB/s) - wget-1.5.3.tar.gz

2. Download file with different name

Using -O (uppercase) option, downloads file with different file name. Here we have given wget.zip file name as show below.

# wget -O wget.zip http://ftp.gnu.org/gnu/wget/wget-1.5.3.tar.gz

--2012-10-02 11:55:54--  http://ftp.gnu.org/gnu/wget/wget-1.5.3.tar.gz
Resolving ftp.gnu.org... 208.118.235.20, 2001:4830:134:3::b
Connecting to ftp.gnu.org|208.118.235.20|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 446966 (436K) [application/x-gzip]
Saving to: wget.zip
100%[===================================================================================>] 446,966     60.0K/s   in 7.5s
2012-10-02 11:56:02 (58.5 KB/s) - wget.zip

3. Download multiple file with http and ftp protocol

Here we see how to download multiple files using HTTP and FTP protocol with wget command at ones.

# wget http://ftp.gnu.org/gnu/wget/wget-1.5.3.tar.gz ftp://ftp.gnu.org/gnu/wget/wget-1.10.1.tar.gz.sig

--2012-10-02 12:11:16--  http://ftp.gnu.org/gnu/wget/wget-1.5.3.tar.gz
Resolving ftp.gnu.org... 208.118.235.20, 2001:4830:134:3::b
Connecting to ftp.gnu.org|208.118.235.20|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 446966 (436K) [application/x-gzip]
Saving to: wget-1.5.3.tar.gz

100%[===================================================================================>] 446,966     56.7K/s   in 7.6s

2012-10-02 12:11:29 (57.1 KB/s) - wget-1.5.3.tar.gz

--2012-10-02 12:11:29--  ftp://ftp.gnu.org/gnu/wget/wget-1.10.1.tar.gz.sig
           => wget-1.10.1.tar.gz.sig

Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /gnu/wget ... done.
==> SIZE wget-1.10.1.tar.gz.sig ... 65
==> PASV ... done.    ==> RETR wget-1.10.1.tar.gz.sig ... done.
Length: 65 (unauthoritative)

100%[===================================================================================>] 65          --.-K/s   in 0s

2012-10-02 12:11:33 (2.66 MB/s) - wget-1.10.1.tar.gz.sig

FINISHED --2012-10-02 12:11:33--
Downloaded: 2 files, 437K in 7.6s (57.1 KB/s)

4. Read URL’s from a file

You can store number of URL’s in text file and download them with -i option. Below we have created tmp.txt under wget directory where we put series of URL’s to download.

# wget -i /wget/tmp.txt

--2012-10-02 12:34:12--  http://ftp.gnu.org/gnu/wget/wget-1.5.3.tar.gz ftp://ftp.gnu.org/gnu/wget/wget-1.10.1.tar.gz.sig
Resolving ftp.gnu.org... 208.118.235.20, 2001:4830:134:3::b
Connecting to ftp.gnu.org|208.118.235.20|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 446966 (436K) [application/x-gzip]
Saving to: wget-1.10.1.tar.gz.sig

100%[===================================================================================>] 446,966     35.0K/s   in 10s

2012-10-02 12:34:23 (42.7 KB/s) - wget-1.10.1.tar.gz.sig

--2012-10-02 12:34:23--  http://mirrors.hns.net.in/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-LiveDVD.iso
Resolving mirrors.hns.net.in... 111.91.91.34, 2401:4800::111:91:91:34
Connecting to mirrors.hns.net.in|111.91.91.34|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1761607680 (1.6G) [application/octet-stream]
Saving to: CentOS-6.3-x86_64-LiveDVD.iso

 45%[==========================================                                          ] 1,262,000   51.6K/s  eta 8h 17m

5. Resume uncompleted download

In case of big file download, it may happen sometime to stop download in that case we can resume download the same file where it was left off with -c option. But when you start download file without specifying -c option wget will add .1 extension at the end of file, considering as a fresh download. So, it’s good practice to add -c switch when you download big files.

# wget -c http://mirrors.hns.net.in/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-LiveDVD.iso

--2012-10-02 12:46:57--  http://mirrors.hns.net.in/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-LiveDVD.iso
Resolving mirrors.hns.net.in... 111.91.91.34, 2401:4800::111:91:91:34
Connecting to mirrors.hns.net.in|111.91.91.34|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 1761607680 (1.6G), 1758132697 (1.6G) remaining [application/octet-stream]
Saving to: CentOS-6.3-x86_64-LiveDVD.iso

 51% [=================================================                                  ] 3,877,262   47.0K/s  eta 10h 27m ^

6. Download file with appended .1 in file name

When you start download without -c option wget add .1 at the end of file and start with fresh download. If .1 already exist .2 append at the end of file.

# wget http://mirrors.hns.net.in/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-LiveDVD.iso

--2012-10-02 12:50:49--  http://mirrors.hns.net.in/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-LiveDVD.iso
Resolving mirrors.hns.net.in... 111.91.91.34, 2401:4800::111:91:91:34
Connecting to mirrors.hns.net.in|111.91.91.34|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1761607680 (1.6G) [application/octet-stream]
Saving to: CentOS-6.3-x86_64-LiveDVD.iso.1

 18% [==================                                                                 ] 172,436     59.2K/s   

See the example files with .1 extension appended at the end of the file.

# ls -l CentOS*

-rw-r--r--. 1 root root 3877262 Oct  2 12:47 CentOS-6.3-x86_64-LiveDVD.iso
-rw-r--r--. 1 root root  181004 Oct  2 12:50 CentOS-6.3-x86_64-LiveDVD.iso.1

7. Download files in background

With -b option you can send download in background immediately after download start and logs are written in /wget/log.txt file.

# wget -b /wget/log.txt ftp://ftp.iinet.net.au/debian/debian-cd/6.0.5/i386/iso-dvd/debian-6.0.5-i386-DVD-1.iso

Continuing in background, pid 3550.

8. Restrict download speed limits

With Option –limit-rate=100k, the download speed limit is restricted to 100k and the logs will be created under /wget/log.txt as shown below.

# wget -c --limit-rate=100k  /wget/log.txt ftp://ftp.iinet.net.au/debian/debian-cd/6.0.5/i386/iso-dvd/debian-6.0.5-i386-DVD-1.iso

/wget/log.txt: Scheme missing.
--2012-10-02 13:16:21--  ftp://ftp.iinet.net.au/debian/debian-cd/6.0.5/i386/iso-dvd/debian-6.0.5-i386-DVD-1.iso
           => debian-6.0.5-i386-DVD-1.iso
esolving ftp.iinet.net.au... 203.0.178.32
Connecting to ftp.iinet.net.au|203.0.178.32|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /debian/debian-cd/6.0.5/i386/iso-dvd ... done.
==> SIZE debian-6.0.5-i386-DVD-1.iso ... 4691312640
==> PASV ... done.    ==> REST 2825236 ... done.
==> RETR debian-6.0.5-i386-DVD-1.iso ... done.
Length: 4688487404 (4.4G), 4685662168 (4.4G) remaining (unauthoritative)

 0% [                                                                                    ] 3,372,160   35.5K/s  eta 28h 39m

9. Restricted FTP and HTTP downloads with username and password

With Options –http-user=username, –http-password=password & –ftp-user=username, –ftp-password=password, you can download password restricted HTTP or FTP sites as shown below.

# wget --http-user=narad --http-password=password http://mirrors.hns.net.in/centos/6.3/isos/x86_64/CentOS-6.3-x86_64-LiveDVD.iso
# wget --ftp-user=narad --ftp-password=password ftp://ftp.iinet.net.au/debian/debian-cd/6.0.5/i386/iso-dvd/debian-6.0.5-i386-DVD-1.iso

10. Find wget version and help

With Options –version and –help you can view version and help as needed.

# wget --version

# wget --help

In this article we have covered Linux wget command with options for daily administrative task. Do man wget if you wan to know more about it. Kindly share through our comment box or if we’ve missed out anything, do let us know.

Source: tecmint.com

Be the first to comment

Leave a Reply

Your email address will not be published.


*