How to automate login a website – Java example

login-form

In this example, we will show you how to login a website via standard Java HttpsURLConnection. This technique should be working in most of the login form.

Tools & Java Library used in this example

  1. Google Chrome Browser – Network tab to analyze HTTP request and response header fields.
  2. jsoup library – Extracts HTML form values.
  3. JDK 6.

1. Analyze Http Headers, form data.

To login a website, you need to know following values :

  1. Login form URL.
  2. Login form data.
  3. URL for authentication.
  4. Http request / response header.

Uses Google Chrome to get above data. In Chrome, right click everywhere, choose “Inspect Element” -> “Network” tab.

chrome-network

Before you code, try login via Chrome, observe how the HTTP request, response and form data works, later you need to simulate the same steps in Java.

2. HttpsURLConnection Example

In this example, we show you how to login Gmail.

Summary :

  1. Send an HTTP “GET” request to Google login form – https://accounts.google.com/ServiceLoginAuth
  2. Analyze the form data via Google Chrome’s “Network” feature. Alternatively, you can view the HTML source code.
  3. Use jSoup library to extract all visible and hidden form’s data, replace with your username and password.
  4. Send a HTTP “POST” request back to login form, along with the constructed parameters
  5. After user authenticated, send another HTTP “GET” request to Gmail page. https://mail.google.com/mail/
Note
This example is just to show you the capability and functionality of Java HttpURLConnection. In general, you should use the Google Gmail API to interact with Gmail.
HttpUrlConnectionExample.java

package com.mkyong;

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.CookieHandler;
import java.net.CookieManager;
import java.net.URL;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.List;
import javax.net.ssl.HttpsURLConnection;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

public class HttpUrlConnectionExample {

  private List<String> cookies;
  private HttpsURLConnection conn;

  private final String USER_AGENT = "Mozilla/5.0";

  public static void main(String[] args) throws Exception {

	String url = "https://accounts.google.com/ServiceLoginAuth";
	String gmail = "https://mail.google.com/mail/";

	HttpUrlConnectionExample http = new HttpUrlConnectionExample();

	// make sure cookies is turn on
	CookieHandler.setDefault(new CookieManager());

	// 1. Send a "GET" request, so that you can extract the form's data.
	String page = http.GetPageContent(url);
	String postParams = http.getFormParams(page, "username@gmail.com", "password");

	// 2. Construct above post's content and then send a POST request for
	// authentication
	http.sendPost(url, postParams);

	// 3. success then go to gmail.
	String result = http.GetPageContent(gmail);
	System.out.println(result);
  }

  private void sendPost(String url, String postParams) throws Exception {

	URL obj = new URL(url);
	conn = (HttpsURLConnection) obj.openConnection();

	// Acts like a browser
	conn.setUseCaches(false);
	conn.setRequestMethod("POST");
	conn.setRequestProperty("Host", "accounts.google.com");
	conn.setRequestProperty("User-Agent", USER_AGENT);
	conn.setRequestProperty("Accept",
		"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
	conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
	for (String cookie : this.cookies) {
		conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
	}
	conn.setRequestProperty("Connection", "keep-alive");
	conn.setRequestProperty("Referer", "https://accounts.google.com/ServiceLoginAuth");
	conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
	conn.setRequestProperty("Content-Length", Integer.toString(postParams.length()));

	conn.setDoOutput(true);
	conn.setDoInput(true);

	// Send post request
	DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
	wr.writeBytes(postParams);
	wr.flush();
	wr.close();

	int responseCode = conn.getResponseCode();
	System.out.println("\nSending 'POST' request to URL : " + url);
	System.out.println("Post parameters : " + postParams);
	System.out.println("Response Code : " + responseCode);

	BufferedReader in = 
             new BufferedReader(new InputStreamReader(conn.getInputStream()));
	String inputLine;
	StringBuffer response = new StringBuffer();

	while ((inputLine = in.readLine()) != null) {
		response.append(inputLine);
	}
	in.close();
	// System.out.println(response.toString());

  }

  private String GetPageContent(String url) throws Exception {

	URL obj = new URL(url);
	conn = (HttpsURLConnection) obj.openConnection();

	// default is GET
	conn.setRequestMethod("GET");

	conn.setUseCaches(false);

	// act like a browser
	conn.setRequestProperty("User-Agent", USER_AGENT);
	conn.setRequestProperty("Accept",
		"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
	conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
	if (cookies != null) {
		for (String cookie : this.cookies) {
			conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
		}
	}
	int responseCode = conn.getResponseCode();
	System.out.println("\nSending 'GET' request to URL : " + url);
	System.out.println("Response Code : " + responseCode);

	BufferedReader in = 
            new BufferedReader(new InputStreamReader(conn.getInputStream()));
	String inputLine;
	StringBuffer response = new StringBuffer();

	while ((inputLine = in.readLine()) != null) {
		response.append(inputLine);
	}
	in.close();

	// Get the response cookies
	setCookies(conn.getHeaderFields().get("Set-Cookie"));

	return response.toString();

  }

  public String getFormParams(String html, String username, String password)
		throws UnsupportedEncodingException {

	System.out.println("Extracting form's data...");

	Document doc = Jsoup.parse(html);

	// Google form id
	Element loginform = doc.getElementById("gaia_loginform");
	Elements inputElements = loginform.getElementsByTag("input");
	List<String> paramList = new ArrayList<String>();
	for (Element inputElement : inputElements) {
		String key = inputElement.attr("name");
		String value = inputElement.attr("value");

		if (key.equals("Email"))
			value = username;
		else if (key.equals("Passwd"))
			value = password;
		paramList.add(key + "=" + URLEncoder.encode(value, "UTF-8"));
	}

	// build parameters list
	StringBuilder result = new StringBuilder();
	for (String param : paramList) {
		if (result.length() == 0) {
			result.append(param);
		} else {
			result.append("&" + param);
		}
	}
	return result.toString();
  }

  public List<String> getCookies() {
	return cookies;
  }

  public void setCookies(List<String> cookies) {
	this.cookies = cookies;
  }

}

Output


Sending 'GET' request to URL : https://accounts.google.com/ServiceLoginAuth
Response Code : 200
Extracting form data...

Sending 'POST' request to URL : https://accounts.google.com/ServiceLoginAuth
Post parameters : dsh=-293322094146108856&GALX=CExqdUbvEr4&timeStmp=&secTok=&_utf8=%E2%98%83
&bgresponse=js_disabled&Email=username&Passwd=password&signIn=Sign+in&PersistentCookie=yes&rmShown=1
Response Code : 200

Sending 'GET' request to URL : https://mail.google.com/mail/
Response Code : 200
<!-- gmail page content.....-->
Note
Refer to this equivalent example, but using Apache HttpClient to send HTTP request.

References

  1. Automated Login To a Website
  2. Android documentation – HttpURLConnection
  3. Sending HTTP Request GET/POST to form with Java?
  4. jSoup library
  5. Apache HttpClient examples

About the Author

author image
mkyong
Founder of Mkyong.com, love Java and open source stuff. Follow him on Twitter. If you like my tutorials, consider make a donation to these charities.

Comments

avatar
61 Comment threads
20 Thread replies
2 Followers
 
Most reacted comment
Hottest comment thread
65 Comment authors
ChaitaliYong KungleMakavelikavithayg6fyf66tfg Recent comment authors
newest oldest most voted
pratik
Guest
pratik

i get a null pointer error for cookies in line for(….,this.cookies)

Chaitali
Guest
Chaitali

Could you solve this? I am also facing the same problem

johan
Guest
johan

thank you so much for the tutorial. i just clicked your ads to support you. ;)

kalid
Guest
kalid

lol

Noname
Guest
Noname

I’m pretty sure this block of code in “GetPageContent()” is useless:

if (cookies != null) {
for (String cookie : this.cookies) {
conn.addRequestProperty(“Cookie”, cookie.split(“;”, 1)[0]);
}
}

Bora
Guest
Bora

Hello,thanks for this tutorial. I am trying to implement this code to my android application to provide automatic login to a specific website. However I am having difficulties in seeing the output on emulator(or on my phone when I run the app) in android studio. Can you please help? I would like to get a specific data field from the page that the user have login.
Thanks in advance.

navin kumar
Guest
navin kumar

After running this code i am not able see any browser with my gmail UI. My requirement is from java code open a browser with gmail UI.

Aparna
Guest
Aparna

Hi Navin,

My requirement is same. It didn’t work for me 2. If you have found any solution please let me know.

Thanks

jay glass
Guest
jay glass

Hi Guys, old post but to answer you’re question – you could either use java to launch a process like this (and you should also be able to add url to end of iexplore.exe):
Runtime rt = Runtime.getRuntime();
Process p = rt.exec(“C:/Program Files/Internet Explorer/IEXPLORE.EXE”);
System.out.println(“P:”+p);
p.waitFor();

System.out.println(“P Now:”+p);

Or this looks good from Stack Overflow:
http://stackoverflow.com/questions/5226212/how-to-open-the-default-webbrowser-using-java

java.awt.Desktop is the class you’re looking for.

import java.awt.Desktop;
import java.net.URI;

// …

if(Desktop.isDesktopSupported())
{
Desktop.getDesktop().browse(new URI(“http://www.example.com”));
}

Thanks Tim Cooper.

Hope this helps anyone

Avinash Sharma
Guest
Avinash Sharma

Hello I am trying to do with a website which allows you to sign up at the same time so I get a Error like sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) and it shows error response code 400

Kostya
Guest
Kostya

All works great! Thanks you!

Eric
Guest
Eric

Hello
it does not work for Android 4.4.2.

java.net.protocolexception: content-length promised 232 bytes, but received 0 on conn.getResponseCode();

use the Apache version instead

Adi
Guest
Adi

sir can you please tell me how to scratch data dynamically from any website that dont have an API

sky
Guest
sky

i tried to user this code for http://www.naver.com

but i got a problem that “Due to security reasons, you need to enter the code in the image along with your password.”

i don’t know
can anybody help me?

ComputerNinja
Guest
ComputerNinja

I dunno if you figured it out yet or not but, the “code in the image” is literally meant to keep bots/programs from logging on. In short, you can’t do it as far as I know.

ComputerNinja
Guest
ComputerNinja

This is great code! worked on first try. If you trying to log into a different website, you need to tweak some of the code. For example, finding the form tags is probably different, it may not even have an Id so you have to change how it finds the appropriate form element. Also, when it’s replicating what a browser does you need to change the referrer. Another one is in the form tag that gets the user and password, the value of the name tags might be different. It could be email instead of username, or it could be… Read more »

Chaitali
Guest
Chaitali

Hello,
The cookie does not generate for me. I have made below changes. However, it still does not work.

Can you kindly help.

String key = inputElement.attr(“id”);
String value = inputElement.attr(“value”);

if (key.equals(“Email”))
value = username;
else if (key.equals(“Passwd-hidden”))
value = password;

Raj
Guest
Raj

Hi MKyong!

Thanks for your tutorial!

I have tried using your code for some other site. But while reading the response, it return unknown forat like “‹” unknown format..,

Please guid me on this..,

naxx
Guest
naxx

Hello, is possible to HttpURLConnection instance from last HTTP GET (gmail content page) interpreted and show in WEB browser.
I want to use Spring MVC, but I don´t know how I can this last HttpURLConnection instance interpreted as VIEW.

Thanks for your answers.

Nimer Shadida Johansson
Guest
Nimer Shadida Johansson

Hi! I am trying to modify this to match login for facebook. I get everything to work, except that it doesn’t go past the login page. My best guess is that it is coded differently for the getFormParams – that the input method that I am trying to use with this code is not compatible for Facebooks loginform and inputElements.

As I do not fully understand how these element(s) work I am hoping that you could nod me in the right direction of what I need to look out for in order to properly modify the code.

Best Regards,
Nimer.

ElMoMo
Guest
ElMoMo

I got 200 Ok regardless if I used your fake ID/password or my own ID/password? How can you verify that I am truly logged in?

Christopher Jazinski
Guest
Christopher Jazinski

How would I be able to redirect using the browser after the authentication/validation has occurred?

Renan Gomes
Guest
Renan Gomes

Thanks, works nice for me! :)

Aparna
Guest
Aparna

Hi Mkyong,

I tried the above code. But it did not open a new browser window or open in the same browser window. It just printed in the console. I need to automatically login into a different web site from my company web site. Please let me know what could be the problem.

Thanks

Yaswanth Kumar
Guest
Yaswanth Kumar

Hi
In the getPageContent method, when is the connection established exactly?

S ab
Guest
S ab

How to use in android? I want to show webpage in webview. I’m begginer plz help me

Avinash Sharma
Guest
Avinash Sharma

This is the bunch of error i get

at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)

at java.lang.reflect.Constructor.newInstance(Unknown Source)

at sun.net.www.protocol.http.HttpURLConnection$10.run(Unknown Source)

at sun.net.www.protocol.http.HttpURLConnection$10.run(Unknown Source)

at java.security.AccessController.doPrivileged(Native Method)

at sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unknown Source)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)

at AutomaticLogin.sendPost(AutomaticLogin.java:122)

at AutomaticLogin.main(AutomaticLogin.java:56)

Caused by: java.io.IOException: Server returned HTTP response code: 400 for URL: https://accounts.craigslist.org/login

at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)

at java.net.HttpURLConnection.getResponseCode(Unknown Source)

at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)

at AutomaticLogin.sendPost(AutomaticLogin.java:116)

… 1 more

Jorge
Guest
Jorge

Hi,

This code works perfectly for google, so…i tried to do the same for facebook, but i can’t establish the connection. How it could be?

anyway, dude…you’re amazing!

Thanks!

Push Paganan
Guest
Push Paganan

this code didnt work for me!Its just shows the google login page!Can you pls help me!!

Daralis
Guest
Daralis

Hi, Did you finally get it work? I got the same result as showing the google login page ..

nanyan
Guest
nanyan

how to go to next page using this?

Jon
Guest
Jon

Thank you for your example, works very well and easy to modify for other websites.

Anju Periwal
Guest
Anju Periwal

I tried your code & it worked fine for google.

However, when I used it for different website, user-id & pwd are not getting posted though response code is 200.
The cookie is not getting generated.
The html read after posting the parameters is of original login page.

However, when I manually fill & submit the form, next page appears & cookie too gets generated.

What can be possible sources of problem?

Lesley
Guest
Lesley

have you figured this out yet? i’m encountering the same problem

Dahn
Guest
Dahn

Great! Thanks a lot for your tutorial.

amir
Guest
amir

With minor changes i figured out how to login to another page that did not provide any api, Thanks!!

pravat
Guest
pravat

HI amir,

Is this code working for you? If yes then please share your code with me.
my id: pravat@eginnovations.com

thanks in advance,
pravat kumar dash

jDave The Magical
Guest
jDave The Magical

I’m trying this with schwab’s website. There are hidden parameters…. how do I modify what I can’t see?

AnkitB
Guest
AnkitB

Hi
I tried to use this code but it agains ask to sign in.
Output:
HTML of gmail login page

I tried to use your code
Guest
I tried to use your code

I tried to use your code

I tried to use your code
Guest
I tried to use your code

tried to use your code • 2 minutes ago ? I tried to use your code • 2 minutes ago ? + I tried to use your code see more 0 0 You must sign in to down-vote this post. • Reply • Share › Twitter Facebook Link Renato Cohen • 2 months ago ? + Hello, I tried to use your code but I got an issue about certification. So I add part of a code to trust all certifications: // Configure the SSLContext with a TrustManager SSLContext ctx = SSLContext.getInstance(“SSL”); TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager()… Read more »