Main Tutorials

How to automate login a website – Java example

login-form

In this example, we will show you how to login a website via standard Java HttpsURLConnection. This technique should be working in most of the login form.

Tools & Java Library used in this example

  1. Google Chrome Browser – Network tab to analyze HTTP request and response header fields.
  2. jsoup library – Extracts HTML form values.
  3. JDK 6.

1. Analyze Http Headers, form data.

To login a website, you need to know following values :

  1. Login form URL.
  2. Login form data.
  3. URL for authentication.
  4. Http request / response header.

Uses Google Chrome to get above data. In Chrome, right click everywhere, choose “Inspect Element” -> “Network” tab.

chrome-network

Before you code, try login via Chrome, observe how the HTTP request, response and form data works, later you need to simulate the same steps in Java.

2. HttpsURLConnection Example

In this example, we show you how to login Gmail.

Summary :

  1. Send an HTTP “GET” request to Google login form – https://accounts.google.com/ServiceLoginAuth
  2. Analyze the form data via Google Chrome’s “Network” feature. Alternatively, you can view the HTML source code.
  3. Use jSoup library to extract all visible and hidden form’s data, replace with your username and password.
  4. Send a HTTP “POST” request back to login form, along with the constructed parameters
  5. After user authenticated, send another HTTP “GET” request to Gmail page. https://mail.google.com/mail/
Note
This example is just to show you the capability and functionality of Java HttpURLConnection. In general, you should use the Google Gmail API to interact with Gmail.
HttpUrlConnectionExample.java

package com.mkyong;

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.CookieHandler;
import java.net.CookieManager;
import java.net.URL;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.List;
import javax.net.ssl.HttpsURLConnection;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

public class HttpUrlConnectionExample {

  private List<String> cookies;
  private HttpsURLConnection conn;

  private final String USER_AGENT = "Mozilla/5.0";

  public static void main(String[] args) throws Exception {

	String url = "https://accounts.google.com/ServiceLoginAuth";
	String gmail = "https://mail.google.com/mail/";

	HttpUrlConnectionExample http = new HttpUrlConnectionExample();

	// make sure cookies is turn on
	CookieHandler.setDefault(new CookieManager());

	// 1. Send a "GET" request, so that you can extract the form's data.
	String page = http.GetPageContent(url);
	String postParams = http.getFormParams(page, "[email protected]", "password");

	// 2. Construct above post's content and then send a POST request for
	// authentication
	http.sendPost(url, postParams);

	// 3. success then go to gmail.
	String result = http.GetPageContent(gmail);
	System.out.println(result);
  }

  private void sendPost(String url, String postParams) throws Exception {

	URL obj = new URL(url);
	conn = (HttpsURLConnection) obj.openConnection();

	// Acts like a browser
	conn.setUseCaches(false);
	conn.setRequestMethod("POST");
	conn.setRequestProperty("Host", "accounts.google.com");
	conn.setRequestProperty("User-Agent", USER_AGENT);
	conn.setRequestProperty("Accept",
		"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
	conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
	for (String cookie : this.cookies) {
		conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
	}
	conn.setRequestProperty("Connection", "keep-alive");
	conn.setRequestProperty("Referer", "https://accounts.google.com/ServiceLoginAuth");
	conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
	conn.setRequestProperty("Content-Length", Integer.toString(postParams.length()));

	conn.setDoOutput(true);
	conn.setDoInput(true);

	// Send post request
	DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
	wr.writeBytes(postParams);
	wr.flush();
	wr.close();

	int responseCode = conn.getResponseCode();
	System.out.println("\nSending 'POST' request to URL : " + url);
	System.out.println("Post parameters : " + postParams);
	System.out.println("Response Code : " + responseCode);

	BufferedReader in = 
             new BufferedReader(new InputStreamReader(conn.getInputStream()));
	String inputLine;
	StringBuffer response = new StringBuffer();

	while ((inputLine = in.readLine()) != null) {
		response.append(inputLine);
	}
	in.close();
	// System.out.println(response.toString());

  }

  private String GetPageContent(String url) throws Exception {

	URL obj = new URL(url);
	conn = (HttpsURLConnection) obj.openConnection();

	// default is GET
	conn.setRequestMethod("GET");

	conn.setUseCaches(false);

	// act like a browser
	conn.setRequestProperty("User-Agent", USER_AGENT);
	conn.setRequestProperty("Accept",
		"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
	conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
	if (cookies != null) {
		for (String cookie : this.cookies) {
			conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
		}
	}
	int responseCode = conn.getResponseCode();
	System.out.println("\nSending 'GET' request to URL : " + url);
	System.out.println("Response Code : " + responseCode);

	BufferedReader in = 
            new BufferedReader(new InputStreamReader(conn.getInputStream()));
	String inputLine;
	StringBuffer response = new StringBuffer();

	while ((inputLine = in.readLine()) != null) {
		response.append(inputLine);
	}
	in.close();

	// Get the response cookies
	setCookies(conn.getHeaderFields().get("Set-Cookie"));

	return response.toString();

  }

  public String getFormParams(String html, String username, String password)
		throws UnsupportedEncodingException {

	System.out.println("Extracting form's data...");

	Document doc = Jsoup.parse(html);

	// Google form id
	Element loginform = doc.getElementById("gaia_loginform");
	Elements inputElements = loginform.getElementsByTag("input");
	List<String> paramList = new ArrayList<String>();
	for (Element inputElement : inputElements) {
		String key = inputElement.attr("name");
		String value = inputElement.attr("value");

		if (key.equals("Email"))
			value = username;
		else if (key.equals("Passwd"))
			value = password;
		paramList.add(key + "=" + URLEncoder.encode(value, "UTF-8"));
	}

	// build parameters list
	StringBuilder result = new StringBuilder();
	for (String param : paramList) {
		if (result.length() == 0) {
			result.append(param);
		} else {
			result.append("&" + param);
		}
	}
	return result.toString();
  }

  public List<String> getCookies() {
	return cookies;
  }

  public void setCookies(List<String> cookies) {
	this.cookies = cookies;
  }

}

Output


Sending 'GET' request to URL : https://accounts.google.com/ServiceLoginAuth
Response Code : 200
Extracting form data...

Sending 'POST' request to URL : https://accounts.google.com/ServiceLoginAuth
Post parameters : dsh=-293322094146108856&GALX=CExqdUbvEr4&timeStmp=&secTok=&_utf8=%E2%98%83
&bgresponse=js_disabled&Email=username&Passwd=password&signIn=Sign+in&PersistentCookie=yes&rmShown=1
Response Code : 200

Sending 'GET' request to URL : https://mail.google.com/mail/
Response Code : 200
<!-- gmail page content.....-->
Note
Refer to this equivalent example, but using Apache HttpClient to send HTTP request.

References

  1. Automated Login To a Website
  2. Android documentation – HttpURLConnection
  3. Sending HTTP Request GET/POST to form with Java?
  4. jSoup library
  5. Apache HttpClient examples

About Author

author image
Founder of Mkyong.com, love Java and open source stuff. Follow him on Twitter. If you like my tutorials, consider make a donation to these charities.

Comments

Subscribe
Notify of
79 Comments
Most Voted
Newest Oldest
Inline Feedbacks
View all comments
pratik
10 years ago

i get a null pointer error for cookies in line for(….,this.cookies)

Chaitali
4 years ago
Reply to  pratik

Could you solve this? I am also facing the same problem

johan
6 years ago

thank you so much for the tutorial. i just clicked your ads to support you. 😉

kalid
5 years ago
Reply to  johan

lol

Noname
5 years ago

I’m pretty sure this block of code in “GetPageContent()” is useless:

if (cookies != null) {
for (String cookie : this.cookies) {
conn.addRequestProperty(“Cookie”, cookie.split(“;”, 1)[0]);
}
}

ComputerNinja
6 years ago

This is great code! worked on first try. If you trying to log into a different website, you need to tweak some of the code. For example, finding the form tags is probably different, it may not even have an Id so you have to change how it finds the appropriate form element. Also, when it’s replicating what a browser does you need to change the referrer. Another one is in the form tag that gets the user and password, the value of the name tags might be different. It could be email instead of username, or it could be name, or pword instead of password. You have to check the source or inspect the form element to find out what they are specifically for that site.

Chaitali
4 years ago
Reply to  ComputerNinja

Hello,
The cookie does not generate for me. I have made below changes. However, it still does not work.

Can you kindly help.

String key = inputElement.attr(“id”);
String value = inputElement.attr(“value”);

if (key.equals(“Email”))
value = username;
else if (key.equals(“Passwd-hidden”))
value = password;

Bora
6 years ago

Hello,thanks for this tutorial. I am trying to implement this code to my android application to provide automatic login to a specific website. However I am having difficulties in seeing the output on emulator(or on my phone when I run the app) in android studio. Can you please help? I would like to get a specific data field from the page that the user have login.
Thanks in advance.

navin kumar
8 years ago

After running this code i am not able see any browser with my gmail UI. My requirement is from java code open a browser with gmail UI.

jay glass
7 years ago
Reply to  navin kumar

Hi Guys, old post but to answer you’re question – you could either use java to launch a process like this (and you should also be able to add url to end of iexplore.exe):
Runtime rt = Runtime.getRuntime();
Process p = rt.exec(“C:/Program Files/Internet Explorer/IEXPLORE.EXE”);
System.out.println(“P:”+p);
p.waitFor();

System.out.println(“P Now:”+p);

Or this looks good from Stack Overflow:
http://stackoverflow.com/questions/5226212/how-to-open-the-default-webbrowser-using-java

java.awt.Desktop is the class you’re looking for.

import java.awt.Desktop;
import java.net.URI;

// …

if(Desktop.isDesktopSupported())
{
Desktop.getDesktop().browse(new URI(“http://www.example.com”));
}

Thanks Tim Cooper.

Hope this helps anyone

Aparna
8 years ago
Reply to  navin kumar

Hi Navin,

My requirement is same. It didn’t work for me 2. If you have found any solution please let me know.

Thanks

Avinash Sharma
8 years ago

Hello I am trying to do with a website which allows you to sign up at the same time so I get a Error like sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) and it shows error response code 400

Kostya
8 years ago

All works great! Thanks you!

Eric
9 years ago

Hello
it does not work for Android 4.4.2.

java.net.protocolexception: content-length promised 232 bytes, but received 0 on conn.getResponseCode();

use the Apache version instead

Adi
9 years ago

sir can you please tell me how to scratch data dynamically from any website that dont have an API

sky
9 years ago

i tried to user this code for http://www.naver.com

but i got a problem that “Due to security reasons, you need to enter the code in the image along with your password.”

i don’t know
can anybody help me?

ComputerNinja
6 years ago
Reply to  sky

I dunno if you figured it out yet or not but, the “code in the image” is literally meant to keep bots/programs from logging on. In short, you can’t do it as far as I know.

Debmalya Jash
10 years ago

Thanks for this tutorial.

I tried to follow this code. Got exception from the following line.

conn = (HttpsURLConnection) obj.openConnection();

Exception is java.lang.ClassCastException: sun.net.www.protocol.http.HttpURLConnection cannot be cast to javax.net.ssl.HttpsURLConnection.

Debmalya Jash
10 years ago
Reply to  Debmalya Jash

java.net.URL obj = new URL(null, url,new sun.net.www.protocol.https.Handler());

solved the problem

Debmalya Jash
10 years ago
Reply to  Debmalya Jash

watch more about this problem in http://stackoverflow.com/a/12796611

fox
1 year ago

hello Great work but how would i login in to a .onion webpage

Bhaskar
3 years ago

1.Type mismatch: cannot convert from org.jsoup.select.Elements to javax.lang.model.util.Elements
Elements inputElements = loginform.getElementsByTag(“input”);
2.Can only iterate over an array or an instance of java.lang.Iterable
for (Element inputElement : inputElements) {
(line # is approx 143-145
I got these two errors. I used maven dependency
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.13.1</version>
</dependency>

Yong Kungle
4 years ago

If you like my tutorials, consider make a donation to

Yong Kungle
4 years ago

If you like my tutorials, consider make a donation

Makaveli
4 years ago

Exception in thread “main” java.lang.ClassCastException: class sun.net.www.protocol.http.HttpURLConnection cannot be cast to class javax.net.ssl.HttpsURLConnection (sun.net.www.protocol.http.HttpURLConnection and javax.net.ssl.HttpsURLConnection are in module java.base of loader ‘bootstrap’)

kavitha
5 years ago

The “name” attribute for password is not present in html .Try changing the code like this
for (Element inputElement : inputElements) {
String key = inputElement.attr(“id”);
String value = inputElement.attr(“value”);

if (key.equals(“Email”))
value = username;
else if (key.equals(“Passwd-hidden”))
value = password;
paramList.add(key + “=” + URLEncoder.encode(value, “UTF-8”));
}

It works for me!

yg6fyf66tfg
5 years ago

please sub to my youtube channel it is nataya quintanilla

yg6fyf66tfg
5 years ago

thanks

mango
5 years ago

im working in blueJ and not able to compile this code

youssef
5 years ago

hi
thanks for the blog,
the mistake in my code is that i not enabled cookies, and after i see your code i try it and it’s work thanks

Raj
6 years ago

Hi MKyong!

Thanks for your tutorial!

I have tried using your code for some other site. But while reading the response, it return unknown forat like “‹” unknown format..,

Please guid me on this..,

naxx
7 years ago

Hello, is possible to HttpURLConnection instance from last HTTP GET (gmail content page) interpreted and show in WEB browser.
I want to use Spring MVC, but I don´t know how I can this last HttpURLConnection instance interpreted as VIEW.

Thanks for your answers.

Nimer Shadida Johansson
7 years ago

Hi! I am trying to modify this to match login for facebook. I get everything to work, except that it doesn’t go past the login page. My best guess is that it is coded differently for the getFormParams – that the input method that I am trying to use with this code is not compatible for Facebooks loginform and inputElements.

As I do not fully understand how these element(s) work I am hoping that you could nod me in the right direction of what I need to look out for in order to properly modify the code.

Best Regards,
Nimer.

ElMoMo
7 years ago

I got 200 Ok regardless if I used your fake ID/password or my own ID/password? How can you verify that I am truly logged in?

Christopher Jazinski
7 years ago

How would I be able to redirect using the browser after the authentication/validation has occurred?

Renan Gomes
8 years ago

Thanks, works nice for me! 🙂

Aparna
8 years ago

Hi Mkyong,

I tried the above code. But it did not open a new browser window or open in the same browser window. It just printed in the console. I need to automatically login into a different web site from my company web site. Please let me know what could be the problem.

Thanks

Yaswanth Kumar
8 years ago

Hi
In the getPageContent method, when is the connection established exactly?