Remove duplicate words from String in Java example shows how to remove duplicate words from String in Java. The example also shows various approaches to do the same.
How to remove duplicate words from String in Java?
Consider below given string value.
|
1 |
String str = "The first second was alright but the second second was tough."; |
Below given example shows how to remove duplicate words from String.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
package com.javacodeexamples.stringexamples; import java.util.Arrays; import java.util.LinkedHashSet; public class RemoveDuplicateWordsStringExample { public static void main(String[] args) { String str = "The first second was alright but the second second was tough."; System.out.println("Original String: "); System.out.println(str); /* * Since the words are separated by space, * we will split the string by one or more space */ String[] strWords = str.split("\\s+"); //convert String array to LinkedHashSet to remove duplicates LinkedHashSet<String> lhSetWords = new LinkedHashSet<String>( Arrays.asList(strWords) ); //join the words again by space StringBuilder sbTemp = new StringBuilder(); int index = 0; for(String s : lhSetWords){ if(index > 0) sbTemp.append(" "); sbTemp.append(s); index++; } str = sbTemp.toString(); System.out.println("String after removing duplicate words: "); System.out.println(str); } } |
Output
|
1 2 3 4 |
Original String: The first second was alright but the second second was tough. String after removing duplicate words: The first second was alright but the tough. |
Since our string contained words separated by a space, we first split the string by one or more space characters. Once we had all the words in the form of a String array, we converted the String array to LinkedHashSet using the asList method of the Arrays class. Since the Set does not allow duplicate elements, duplicate words were not added to the LinkedHashSet. Once we had all the unique words in LinkedHashSet, we joined them with space to create a string without duplicate words.
As we have seen the longer and descriptive version of the program above, it’s time for the shortcut. The above given complete program can be rewritten in just one line.
|
1 2 3 4 5 6 7 |
String str = "The first second was alright but the second second was tough."; str = new LinkedHashSet<String>( Arrays.asList(str.split("\\s+")) ).toString().replaceAll("[\\[\\],]", ""); System.out.println("String after removing duplicate words: "); System.out.println(str); |
Output
|
1 2 |
String after removing duplicate words: The first second was alright but the tough. |
Just one line. We clubbed the creation of the LinkedHashSet from the List of words that are created from splitting the string by space. Instead of iterating the Set, we used toString method which returns the set elements in “[e1, e2, e3, ..]” format. Since “[“, “]” and “,” were not needed, we replaced them with an empty string.
How to remove duplicate words from String using Java 8?
If you are using Java 8, you can do that in just one line of code as given below.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
package com.javacodeexamples.stringexamples; import java.util.*; import java.util.stream.*; public class RemoveDuplicateWordsStringExample { public static void main(String[] args) { String str = "The first second was alright but the second second was tough."; str = Arrays.stream( str.split("\\s+")).distinct().collect(Collectors.joining(" ") ); System.out.println(str); } } |
Output
|
1 |
The first second was alright but the tough. |
Final words: Why we used LinkedHashSet and not HashSet? Well, the simple reason is HashSet does not maintain the order of the elements. So if we used HashSet to store the words, we could not guarantee the order of the words in a sentence once we joined them back using the space. LinkedHashSet maintains the order so after removing duplicates, we got the words in the same order.
This example is a part of Java String tutorial
Please let me know your views in the comments section below.
About the author
- Gson – Serialize HashMap null entries - April 5, 2023
- Gson: Convert String to JsonObject without POJO - April 5, 2023
- Gson – Convert JSON to a Typed ArrayList - April 5, 2023
Hello, I have two homeworks questions in Java at SMC. So the first question is a palindrome integer Number, if the input is 121, the output is 3 (since 121 is a palindrome). But, if input is 12393, then the output is 7 (because 12393 -> 1239321). how would i go about finding this solution.
The second question is how do i remove duplicates. So for example input: KCCK , output is “empty”. If input: ZZZ, output is Z. If the input is: KKCCDD, the output is D.
This is a beginning class (my 1st month in JAVA) but it feels too fast. Thx
Hello Carlos,
Have a trial and error approach. That is the way you learn, by making mistakes.
I can give you a complete solution but that would do more harm than good.
Good luck. Thanks.
you have given a very tough and complex approach, i’ve solved it using simple methods of java..
my code:
import java.util.Scanner;
class Sample{
public static void main(String[] args)
{
Scanner obj=new Scanner(System.in);
String str=obj.nextLine();
String[] abc=str.split(” “);
int len=abc.length;
System.out.println(len);
for(int i=0;i<len;i++)
{
for(int j=i+1;j<len;j++){
if(abc[i].equals(abc[j]))
{
for(int k=j;k<len-1;k++)
{
abc[k]=abc[k+1];
}
len–;
}
}
}
for(int a=0;a<len;a++)
{
System.out.print(abc[a] +" ");
}
}
}
what about the “The” and “the”?
how can we remove that?
Hi,
The quick solution would be to convert the original string to the lower case using toLowerCase method before removing the duplicate words.
Thanks.