Curl With

C

What is the curl command? Learning and testing APIs with ...

What is the curl command? Learning and testing APIs with …

cURL, which stands for client URL, is a command line tool that developers use to transfer data to and from a server. At the most fundamental, cURL lets you talk to a server by specifying the location (in the form of a URL) and the data you want to send. cURL supports several different protocols, including HTTP and HTTPS, and runs on almost every platform. This makes cURL ideal for testing communication from almost any device (as long as it has a command line and network connectivity) from a local server to most edge devices.
The most basic command in curl is curl. The curl command is followed by the URL, from which we would like to retrieve some kind of data. In this case, it would return the html source for
Underlying the curl command is the libcurl development library, which has bindings for almost any codebase.
cURL is also the name of the software project, which encompasses both the curl command-line tool and the libcurl development library.
Prerequisites
To try out the commands in this article, you need a command shell and internet access. We are going to be using the NASA APOD (Astronomy Picture Of the Day) API to create some examples. This API is open source but you will need to sign up for a developer key, which takes just a minute to get signed up.
Why use curl?
So, why should you use cURL? Consider these benefits of this software project:
It is highly portable. It is compatible with almost every operating system and connected device.
It is useful for testing endpoints, to check if they are working.
It can be verbose, providing details of exactly what has been sent/received, which is helpful for debugging.
It has good error logging.
It can be rate limited.
Sending API requests
We can use curl to send API requests. Each request is generally made up of four main parts:
An endpoint, which is the address (URL) to which we are sending the request.
An HTTP method. The most common methods used are GET, POST, PUT and DELETE.
GET is used to retrieve a resource from a server. This could be a file, information, or an image.
POST is used to send information to the server.
PUT can be used to create or update a resource. This could be used to create or update a record in a database or update the contents of a file.
DELETE is used to delete a resource such as a database entry.
These actions for these methods are the recommended actions, but it’s up to the API specification and implementation to define what exactly happens.
Headers, which contain metadata about the request, such as content type, user agent, and so on.
Body, which is the message body and contains the data that we want to send, if any. Generally, the body is used with POST and PUT methods.
curl command options
There are over two hundred curl options. You can see some of them by typing curl -h in a terminal. The most commonly used command options include these:
-I returns only the HTTPS headers
curl –request GET ‘&date=2020-01-01′ -I
This command will return header fields such as Date, Content-Type etc
-v is the verbose option
curl –request GET ‘NASA_API_KEY&date=2020-01-01’ -v
This verbose command will show you everything that happens when you run the curl command, from connection to the headers and any data returned. Here we also get the description of the image that is being returned by the request, along with the image url.
-o stores the output in a file
curl –request GET ‘NASA_API_KEY&date=2020-01-01’ –output curloutput
Combining curl with other CLI commands
Combining curl with other cli commands can be really handy in situations where you want to use the output of a command as the input to a curl command or vice versa.
As an example, you could see if a webpage contains a certain piece of text using curl and grep.
Here is an example of using curl and Python to extract the image link from a request to the NASA API and display it in the Preview app (MacOS only):
curl –request GET “NASA_API_KEY&date=2020-01-01” -s | python3 -c “import sys, json; print(()[‘url’])” | xargs curl -o && open -a Preview
In this example we use curl to make a GET request on the Nasa API endpoint. This returns json data, which we use in a small Python script to extract the url of the image. We then use the curl command to get the image and open it using Preview on the mac.
You don’t have to use the command line curl to make API requests. You can use a number of different tools to interact with an API, such as HTTPie, Postman, and Rest Client in VS Code.
HTTPie
HTTPie is a command-line HTTP client that is touted as more friendly to users. It also includes a more expressive, color-coded UI. They have an online version, which is really neat.
Postman
Postman is a UI-based client for all things related to API development, and it’s arguably one of the most popular. Postman is very powerful.
You can generate and execute curl commands from within Postman. To generate curl commands, you can enter the request URL and parameters, and then click on the code option on the right-hand side:
A box is displayed with the option to select from a number of languages, including curl. Select curl to see the generated curl command.
Postman gives you a history of all the requests that you’ve built and even data-stamps them, which can be nice for collecting data points or references as you work with an API. Think of it as the client taking notes for you.
Rest Client in VS Code
Rest Client for VS Code is probably one of my favorite tools for executing curl commands. It’s lightweight and has good syntax highlighting. It’s a really useful add-on to do some quick curl requests from within VS Code.
You can simply type in your curl command and a ‘send request’ option will appear above.
After you click send request, another tab opens with the response.
Summary and Next Steps
In this article, we introduced you to the basic curl command and its most useful options. We also mentioned only a handful of the tools that are available to help you get started with cURL. Now you can begin using cURL to test your endpoints and troubleshoot your applications.
To find out more about APIs and API Management, check out the following articles and videos available on IBM Developer:
API Fundamentals
What is a REST API
Acknowledgements
This article was originally authored by Amara Graham and published in April of 2019.
15 Tips On How to Use 'Curl' Command in Linux - Tecmint

15 Tips On How to Use ‘Curl’ Command in Linux – Tecmint

Back in the mid-1990’s when the Internet was still in its infancy, a Swedish programmer named Daniel Stenberg started a project that eventually grew into what we know as curl today.
Initially, he aimed at developing a bot that would download currency exchange rates from a web page periodically and would provide Swedish Kronor equivalents in US dollars to IRC users.
Long story short, the project thrived, adding several protocols and features along the way – and the rest is history. Now let’s dive in with both feet and learn how to use curl to transfer data and more in Linux!
We have put together the following list of 15 curl commands for you.
1. View curl Version
The -V or –version options will not only return the version, but also the supported protocols and features in your current version.
$ curl –version
curl 7. 47. 0 (x86_64-pc-linux-gnu) libcurl/7. 0 GnuTLS/3. 4. 10 zlib/1. 2. 8 libidn/1. 32 librtmp/2. 3
Protocols: dict file ftp ftps gopher imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets
2. Download a File
If you want to download a file, you can use curl with the -O or -o options. The former will save the file in the current working directory with the same name as in the remote location, whereas the latter allows you to specify a different filename and/or location.
$ curl -O # Save as
$ curl -o # Save as
3. Resume an Interrupted Download
If a download was interrupted for some reason (for example, using Ctrl + c), you can resume it very easily. The use of -C – (dash C, space dash) tells curl to resume the download beginning where it left off.
$ curl -C – -O
Download File Using Curl Command
4. Download Multiple Files
With the following command you will download and from and, respectively, in one go.
$ curl -O -O
5. Download URLs From a File
If you combine curl with xargs, you can download files from a list of URLs in a file.
$ xargs -n 1 curl -O < Download Multiple Files with Curl 6. Use a Proxy with or without Authentication If you are behind a proxy server listening on port 8080 at, do. $ curl -x -U user:password -O where you can skip -U user:password if your proxy does not require authentication. 7. Query HTTP Headers HTTP headers allow the remote web server to send additional information about itself along with the actual request. This provides the client with details on how the request is being handled. To query the HTTP headers from a website, do: $ curl -I Curl Query HTTP Headers This information is also available in your browser’s developer tools. 8. Make a POST request with Parameters The following command will send the firstName and lastName parameters, along with their corresponding values, to $ curl --data "firstName=John&lastName=Doe" You can use this tip to simulate the behavior of a regular HTML form. 9. Download Files from an FTP Server with or without Authentication If a remote FTP server is expecting connections at ftpyourftpserver, the following command will download in the current working directory. $ curl -u username:password -O ftpyourftpserver/ where you can skip -u username:password if the FTP server allows anonymous logins. 10. Upload Files to an FTP server with or without Authentication To upload a local file named to ftpyourftpserver using curl, do: $ curl -u username:password -T ftpyourftpserver 11. Specify User Agent The user agent is part of the information that is sent along with an HTTP request. This indicates which browser the client used to make the request. Let’s see what our current curl version uses as default, and let’s change it later to “I am a new web browser”: $ curl -I localhost --user-agent "I am a new web browser" Curl Check User Agent 12. Store Website Cookies Want to see which cookies are downloaded to your computer when you browse to? Use the following command to save them to You can then use cat command to view the file. $ curl --cookie-jar -O Curl Store Website Cookies 13. Send Website Cookies You can use the cookies retrieved in the last tip in subsequent requests to the same site. $ curl --cookie 14. Modify Name Resolution If you’re a web developer and want to test a local version of before pushing it live, you can make curl resolve to your localhost like so: $ curl --resolve Thus, the query to will tell curl to request the site from localhost instead of using DNS or the /etc/hosts file. 15. Limit Download Rate To prevent curl from hosing your bandwidth, you can limit the download rate to 100 KB/s as follows. $ curl --limit-rate 100K -O Summary In this article we have shared a brief history of the origins of curl and explained how to use it through 15 practical examples. Do you know of any other curl commands that we may have missed in this article? Feel free to share them with our community in the comments! Also, if you have questions feel free to let us know. We look forward to hearing from you! If You Appreciate What We Do Here On TecMint, You Should Consider: TecMint is the fastest growing and most trusted community site for any kind of Linux Articles, Guides and Books on the web. Millions of people visit TecMint! to search or browse the thousands of published articles available FREELY to all. If you like what you are reading, please consider buying us a coffee ( or 2) as a token of appreciation. We are thankful for your never ending support. 11 cURL Command Usage with Real-Time Example - Geekflare

11 cURL Command Usage with Real-Time Example – Geekflare

cURL is a command-line tool to get or send data using URL syntax.
If you are working as a developer or in the support function, you must be aware of cURL command usage to troubleshoot web applications. cURL is a cross-platform utility means you can use on Windows, MAC, and UNIX.
The following are some of the most used syntaxes with an example to help you.
Verify if you can connect to the URL
If you are working on a UNIX system and trying to connect the external URL, the first thing you wish to do is to check if you can access the URL through curl.
curl
It will not throw any output. However, if the server can’t connect, then you will get errors such as could not resolve the host.
[[email protected] tmp]# curl
curl: (6) Could not resolve host:; Unknown error
[[email protected] tmp]#
Save URL/URI output to file
If you have to save the URL or URI contents to a specific file, you can use the following syntax
curl >
ex:
[[email protected] tmp]# curl > /tmp/gfhtml% Total% Received% Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 18557 0 18557 0 0 72565 0 –:–:– –:–:– –:–:– 72772
Above example will save all the contents from to /tmp/
Show request and response header
If you are having issues and would like to validate, you are getting the expected request and response header.
curl -v
Ex:
[[email protected] tmp]# curl -v * About to connect() to port 443 (#0)
* Trying 104. 25. 134. 107…
* Connected to (104. 107) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/
CApath: none
* SSL connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject:, OU=PositiveSSL Multi-Domain, OU=Domain Control Validated
* start date: Nov 07 00:00:00 2019 GMT
* expire date: May 15 23:59:59 2020 GMT
* common name:
* issuer: CN=COMODO ECC Domain Validation Secure Server CA 2, O=COMODO CA Limited, L=Salford, ST=Greater Manchester, C=GB
> GET / HTTP/1. 1
> User-Agent: curl/7. 29. 0
> Host:
> Accept: */*
>
< HTTP/1. 1 200 OK < Date: Sat, 09 Nov 2019 19:41:37 GMT < Content-Type: text/html; charset=UTF-8 < Transfer-Encoding: chunked < Connection: keep-alive < Set-Cookie: __cfduid=d2ce6cd359ebc0b6eb5ff3a454ed042021573328497; expires=Sun, 08-Nov-20 19:41:37 GMT; path=/;; HttpOnly; Secure < Vary: Accept-Encoding < Link: <>; rel=”
< Link: <>; rel=shortlink
< X-SRCache-Fetch-Status: HIT < X-SRCache-Store-Status: BYPASS < X-Frame-Options: SAMEORIGIN < X-Powered-By: EasyEngine v4. 0. 12 < Via: 1. 1 google < CF-Cache-Status: DYNAMIC < Strict-Transport-Security: max-age=15552000; preload < X-Content-Type-Options: nosniff < Alt-Svc: h3-23=":443"; ma=86400 < Expect-CT: max-age=604800, report-uri=" < Server: cloudflare < CF-RAY: 533243e4bcd4bbf4-LHR < Download at a limit rate If you are working on optimization and would like to see how much time does it take to download at a particular speed, you can:- curl –-limit-rate 2000B curl –-limit-rate 2000B Using a proxy to connect Very handy if you are working on the DMZ server where you need to connect to the external world using a proxy. curl --proxy yourproxy:port Test URL with injecting header You can use curl by inserting a header with your data to test or troubleshoot the particular issue. Let’s see the following example to request with Content-Type. curl --header 'Content-Type: application/json' By doing above, you are asking curl to pass Content-Type as application/json in the request header. Display only response header If you are doing some troubleshooting and quickly want to check the response header, you can use the following syntax. curl --head Ex: [[email protected] tmp]# curl --head HTTP/1. 1 200 OK Date: Sat, 09 Nov 2019 19:51:23 GMT Content-Type: text/html Connection: keep-alive Set-Cookie: __cfduid=d3cb2c7b8e566ad99c870b0af12b0f1eb1573329083; expires=Sun, 08-Nov-20 19:51:23 GMT; path=/;; HttpOnly X-GUploader-UploadID: AEnB2Uo96JhvJmR2zYUL-Ndh2ta3UD_ykQAB5C7O8cjZQhCf-GxHQ0MsodSzRnl3guSN3ywAYNjtWcPXfwDXjLg3bQ-P5vQMOA Expires: Sat, 09 Nov 2019 20:51:23 GMT Cache-Control: public, max-age=3600 Last-Modified: Mon, 06 Aug 2018 10:45:47 GMT x-goog-generation: 1533552347482034 x-goog-metageneration: 1 x-goog-stored-content-encoding: identity x-goog-stored-content-length: 24620 x-goog-hash: crc32c=DpDPAQ== x-goog-hash: md5=cIP/3rusdUx12Zla1kf1yA== x-goog-storage-class: MULTI_REGIONAL Accept-Ranges: bytes CF-Cache-Status: DYNAMIC Expect-CT: max-age=604800, report-uri=" Server: cloudflare CF-RAY: 53325234dc2fbb9a-LHR Connect HTTPS/SSL URL and ignore any SSL certificate error When you try to access SSL/TLS cert secured URL and if that is having the wrong cert or CN doesn’t match, then you will get the following error. curl: (51) Unable to communicate securely with peer: requested domain name does not match the server's certificate. Good news, you can instruct cURL to ignore the cert error with --insecure flag. curl --insecure Connect using a specific protocol (SSL/TLS) Very handy to test if a particular URL can handshake over specific SSL/TLS protocol. To connect using SSL v3 curl --sslv3 and for different TLS versions curl --tlsv1 curl --tlsv1. 0 curl --tlsv1. 1 curl --tlsv1. 2 curl --tlsv1. 3 Download file from FTP Server You can use curl to download the file as well by specifying username and password. curl -u user:password -O ftpftpurl/ You can always use “-v” with any syntax to print in verbose mode. Using Host Header The host header is useful to test the target URL over IP when the requested content is only available when host header matches. Or, if you want to test application using load balancer IP/URL. curl --header 'Host: ' How about using cURL online? Yes, that’s possible with the following tools. You can execute cURL remotely. Online CURL – a lightweight tool to fetch the URL online and the possibility to add the following options. --connect-timeout --cookie --data --header --head --location --max-time --proxy --request --user --url --user-agent cURL command line builder – this one is different. It helps you to build the curl command where you can enter information in nice UI, and at the bottom, you will get the cURL command. cURL is a useful utility to troubleshoot a real-time connectivity issue, and I hope the above helps you. If you are looking to learn more, then I would recommend Linux Command Line Basics online course.

Frequently Asked Questions about curl with

What does curl do with?

cURL, which stands for client URL, is a command line tool that developers use to transfer data to and from a server. At the most fundamental, cURL lets you talk to a server by specifying the location (in the form of a URL) and the data you want to send.Feb 23, 2021

How do you curl users?

The syntax for the curl command is as follows: curl [options] [URL…] In its simplest form, when invoked without any option, curl displays the specified resource to the standard output. The command will print the source code of the example.com homepage in your terminal window.Nov 27, 2019

What is curl and how do you use it?

cURL is a command-line tool that you can use to transfer data via network protocols. The name cURL stands for ‘Client URL’, and is also written as ‘curl’. This popular command uses URL syntax to transfer data to and from servers. Curl is powered by ‘libcurl’, a free and easy-to-use client-side URL transfer library.Dec 23, 2020

About the author

proxyreview

If you 're a SEO / IM geek like us then you'll love our updates and our website. Follow us for the latest news in the world of web automation tools & proxy servers!

By proxyreview

Recent Posts

Useful Tools