Overview
In this tutorial, you will learn how to use the curl command in Linux and OSX for making GET and POST requests. You will also learn how to send form data, upload files, and traverse proxies.
The curl command is the prevailing utility for working with the Web from a command-line. The utility provides the same functionality as any web browser, without the UI, that enables you to mimic any type of your request done from a browser. In this tutorial, you will learn ways to use Curl effectively from a command-line to perform common requests.
Data can be downloaded from a website, forms can be posted to, typicaly RESTful commands can be tested against applications. The list of possibilities to endless with this popular tool.
You could effectively use curl in place of Postman, for example. However, there are certain niceties about Postman that we recognize, so it depends on what you need out of the tool.
What’s Covered
The following topics are discussed in this tutorial.
- HTTP Get requests
- HTTP Post requests
- Basic HTTP Authentication
- Setting HTTP Headers with curl
- Send curl requests through a proxy server
Simple Examples
The following are commonly used simple curl commands. Use them to make basic requests to web endpoints and APIs.
Simple GET Request
curl http://www.serverlab.ca
Following 301 and 302 Redirects with Curl
By default, redirects will not be followed by the tool . In order to instruct it to follow redirects you must use either the -L or --location flags.
curl -L http://www.serverlab.ca
curl --location http://www.serverlab.ca
Basic HTTP Authentication
curl –basic -u <user> –digest
HTTPS endpoints without verifying TLS
Validation checks will fail when you attempt to curl against a SSL\TLS endpoint that is not publicly trusted. Using the --insecure flag curl will ignore certificate validation issues. This most commonly used with self-signed certificates.
curl --insecure <host>
curl --insecure https://www.serverlab.ca
Send requests through specific network interfaces
Many servers have separate network interfaces attached them to. When you want to curl from a specific network interface, use the --interface flag with the name of the network interface you want to test with.
curl --interface <name> <host>
curl --interface eth0:1 https://www.serverlab.ca
Downloading Files or Outputting Results to File
When data needs to be downloaded from a website, such as downloading a script or binary file, the output needs to be outputted to disk. For these use cases, the -o or –output flags are used with the curl command.
curl http://www.serverlab.ca/downloads/install-script.sh -o install-script.sh
Setting TCP/IP Version for Requests
Requests will automatically use IPv4 or IPv6 depending on the system network configuration. However, there are times were you would want to explicitly set which version should be used. Curl provides the –ipv4 and –ipv6 flags for this purpose.
To send requests via IPv4, use the -4 or –ipv4 flags.
curl -4 http://www.serverlab.ca
curl –ipv4 http://www.serverlab.ca
To send requests via IPv6, use the -6 or –ipv6 flags.
curl -6 http://www.serverlab.ca
curl -ipv6 http://www.serverlab.ca
Post Requests
Not every curl command is a GET request. The following examples show you how to perform POSTS to endpoints, and how to set send data with the command. The -X flag is used to set requests to another type, which in the following examples will be POST.
curl -X POST http://www.severlab.ca/example-form.php
A post without data isn’t very useful. To add data from the command-line the -d option is used. This flag causes curl to pass the content-type value as application/x-www-form-urlencoded.
curl -X POST -d field1=example1 -d field2=example2 http://www.serverlab.ca/api
When posting form field data, the data should be url-encoded. This is done using –data-urlencode flag.
curl -X POST --data-urlencode title="my notes" content="this is a note" http://www.serverlab.com/admin/save-note.php
To interactively send data to the endpoint, the @ symbol is used with the -d flag.
curl -X POST -d=@ http://www.serverlab.ca/example-form.php
Data values can be fetched from a file instead, and similar to posting data interactively you use the @ symbol, but with a file name. When using files line breaks are removed, which allows you nicely format the file for better legibility.
curl -X [email protected] http://www.serverlab.ca/example-form.php
Sending Requests through a Proxy
Many organizations use a forward proxy server to handle all external Internet requests. In environments where a proxy isn’t already set with HTTP_PROXY, for example, or where a separate proxy must be used, the -x flag is used.
Simple Proxy Request
curl -x http://my.proxy.com:8080 http://www.serverlab.ca
Proxy Request with Username and Password
Some proxies require a username and password. This can be done by including the -U or –proxy-user flags to the command.
curl -U <user:password> -x [protocol:]<proxy-server>[:port] <endpoint>
curl -U student:password1 -x https://my.proxy.server.com:80 https://google.com
Conclusion
Curl is a powerful, yet light-weight command-line tool for making requests to web servers and API endpoints. In this tutorial, you have learned how to make simple GET and POST requests, as well as to upload data and send requests through a proxy.
The tool provides a lot more functionality than what is covered in this post. For more information, use the -h flag to display more command and options.