This article shows how to use the -U (uppercase U) option to change the user agent in wget command. Terminal wget -U {user-agent} {location} Change user agent in wget The below command changes the wget user agent to Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7). Terminal wget -U "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)" …
A jsoup HTML parser example to show you how to parse and get all HTML hyperlinks from a web page: pom.xml <dependency> <groupId>org.jsoup</groupId> <artifactId>jsoup</artifactId> <version>1.12.1</version> </dependency> JsoupFindLinkSample.java package com.mkyong; import org.jsoup.Jsoup; import org.jsoup.nodes.Document; import org.jsoup.nodes.Element; import org.jsoup.select.Elements; import java.io.IOException; import java.util.HashSet; import java.util.Set; public class JsoupFindLinkSample { public static void main(String[] args) throws IOException { …
This article shows you how to use the new Java 11 HttpClient APIs to send HTTP GET/POST requests, and some frequent used examples. HttpClient httpClient = HttpClient.newBuilder() .version(HttpClient.Version.HTTP_2) .followRedirects(HttpClient.Redirect.NORMAL) .connectTimeout(Duration.ofSeconds(20)) .proxy(ProxySelector.of(new InetSocketAddress("proxy.yourcompany.com", 80))) .authenticator(Authenticator.getDefault()) .build(); 1. Synchronous Example This example sends a GET request to https://httpbin.org/get, and print out the response header, status code, and …
This article shows you how to use the OkHttp library to send an HTTP GET/POST requests and some frequent used examples. P.S Tested with OkHttp 4.2.2 pom.xml <dependency> <groupId>com.squareup.okhttp3</groupId> <artifactId>okhttp</artifactId> <version>4.2.2</version> </dependency> 1. Synchronous Get Request OkHttpExample1.java package com.mkyong.http; import okhttp3.Headers; import okhttp3.OkHttpClient; import okhttp3.Request; import okhttp3.Response; import java.io.IOException; public class OkHttpExample1 { // only …
Eclipse IDE prompts warning on new DefaultHttpClient, mark this class as deprecated. package com.mkyong.web.controller; import org.apache.http.Header; import org.apache.http.HttpResponse; import org.apache.http.client.HttpClient; import org.apache.http.client.methods.HttpGet; import org.apache.http.impl.client.DefaultHttpClient; public class WebCrawler { public static void main(String[] args) throws Exception { HttpClient client = new DefaultHttpClient(); HttpGet request = new HttpGet("https://mkyong.com"); HttpResponse response = client.execute(request); //… } } Solution Dive …
In this article, we will show you a few examples to make HTTP GET/POST requests via the following APIs Apache HttpClient 4.5.10 OkHttp 4.2.2 Java 11 HttpClient Java 1.1 HttpURLConnection (Not recommend) 1. Apache HttpClient In the old days, this Apache HttpClient is the de facto standard to send an HTTP GET/POST request in Java. …