Jsoup login to a website using post method example shows how to login to a website by posting all parameters using Jsoup. The example also shows how to manage cookies while logging in to the website.
How to login to the website using Jsoup using the POST method?
Step 1:
Have a look at how to post form data using Jsoup example.
Step 2:
Open the website login page in your browser which supports inspecting the HTML elements to make things easier. For this example, we are going to login to http://www.rediff.com.
We need to inspect the login form so that we can determine
1) All required form parameters.
2) Where login form data needs to be submitted.
Here is the login form.
We need to determine the action URL and form data fields. Right click on the login Id text field and select inspect menu item as given in the screenshot.
It will open the developer tools with elements tab opened in the lower half of the browser. Tools will also have a login id input element selected in the source. Now, locate the parent form element of the login id input element.
This is the form we are interested in. URL mentioned in the “action” attribute of the form is where we need to post the data. We also need to send all inputs (including hidden), textarea and select elements contained in the form along with name and values as parameters in the POST method. Make a note of all the parameter names and values.
Once the parameters and action URL is determined, we can go ahead and log in to the website using Jsoup as given below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 |
package com.javacodeexamples.libraries.jsoup; import java.io.IOException; import java.util.HashMap; import java.util.Map; import org.jsoup.Jsoup; import org.jsoup.Connection.Response; import org.jsoup.nodes.Document; public class JsoupLoginExample { public static void main(String[] args) { try { //grab login form page first Response loginPageResponse = Jsoup.connect("https://mail.rediff.com/cgi-bin/login.cgi") .referrer("http://www.rediff.com/") .userAgent("Mozilla/5.0") .timeout(10 * 1000) .followRedirects(true) .execute(); System.out.println("Fetched login page"); //get the cookies from the response, which we will post to the action URL Map<String, String> mapLoginPageCookies = loginPageResponse.cookies(); //lets make data map containing all the parameters and its values found in the form Map<String, String> mapParams = new HashMap<String, String>(); mapParams.put("FormName", "existing"); mapParams.put("seclogin", "on"); mapParams.put("login", "YOUR_USER_ID"); mapParams.put("passwd", "YOUR_PASSWORD"); mapParams.put("remember", "1"); mapParams.put("proceed", "Go"); //URL found in form's action attribute String strActionURL = "https://mail.rediff.com/cgi-bin/login.cgi"; Response responsePostLogin = Jsoup.connect(strActionURL) //referrer will be the login page's URL .referrer("https://mail.rediff.com/cgi-bin/login.cgi") //user agent .userAgent("Mozilla/5.0") //connect and read time out .timeout(10 * 1000) //post parameters .data(mapParams) //cookies received from login page .cookies(mapLoginPageCookies) //many websites redirects the user after login, so follow them .followRedirects(true) .execute(); System.out.println("HTTP Status Code: " + responsePostLogin.statusCode()); //parse the document from response Document document = responsePostLogin.parse(); System.out.println(document); //get the cookies Map<String, String> mapLoggedInCookies = responsePostLogin.cookies(); /* * For all the subsequent requests, you need to send * the mapLoggedInCookies containing cookies */ } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } } } |
Output
|
1 2 3 4 5 6 |
Fetched login page HTTP Status Code: 200 <html> <head> <script language="JavaScript" type="text/javascript"> ... |
Disclaimer: The URLs given/accessed in the example are purely for demonstration purposes. You should not access any URLs in any way which is not permitted by the website’s terms of service agreement.
Please make sure to set proper user agent and referrer before making a connection, most of the web servers reject requests not having proper user agent string and referrer values. If you are behind a proxy server, you may also need to set Jsoup proxy.
If you are unable to determine the action URL and/or parameters, you can use the developer devtools to determine that. First open the login page in a chrome browser, press F12 to open the devtools, and go to the Network tab.
Click the Clear button to clear older requests. Now enter the user id and password and click the login button. The network tab will display the outgoing requests. Click on the relevant raw in the devtools to see the parameters and other details.
This example is a part of the Jsoup tutorial with examples.
Please let me know your views in the comments section below.
About the author
- Gson – Serialize HashMap null entries - April 5, 2023
- Gson: Convert String to JsonObject without POJO - April 5, 2023
- Gson – Convert JSON to a Typed ArrayList - April 5, 2023
Submit button is hidden with javascript.. how to see the value?
Use the inspector/developer options in chrome browser to see hidden fields values (F12 key in Windows).
Thank you rahimv. This is the best tutorial I have found about the subject.
Helped me a lot.
I am glad it could help you. Thanks.
Thank rahimv.This helped me alot.Can you also provide code for logging into gmail account.
Hello Shirisha,
Unfortunately I cannot. You will have to do trial and error to make it work.
One more thing, If you have to fill the captcha before submitting any form, this method does not work. Captcha is designed to prevent such type of automatic submissions by programs.
Best of luck!
Hi Rahimv,
I followed your rediff mail code example to login a website to fetch web contents after successful login.
I am getting HTTP status code as 200, but I am not getting the web contents after successful login.
May I know the reason for this issue or any clues which will help me to fix this issue?
Hi Rajakumar,
There could be many reasons. Is the website redirecting you to another page after login is successful?. I would suggest you to study the complete flow of events and pages using the browser first and then only implement it using Jsoup.
If the login page is returning you the HTTP code 200, then in the same session try to open a webpage that only authenticated users can view (you need to forward the cookies you received).
If that does not work, I suspect you are missing something in the flow. Trial and error is the way to go in that case.
Thank you.
Thank u bro
Glad I could help. Thanks.