Capattax Capattax - 4 months ago 9
Python Question

Checking data in a file for duplicates (Python)

I am trying to make a list of topics for another project to use and I am storing the topics in

Topics.txt
. However, when the topics are stored in the file, I do not want duplicate topics. So when I am saving my topics to my
Topics.txt
file, I also save them to a
Duplicates.txt
file. What I want to do is create a conditional statement that won't add topics to
Topics.txt
if the topics are in the
Duplicates.txt
. My problem is, I don't know how I could create a conditional statement that could check if the topic is listed in
Duplicates.txt
. A problem may arise if you scan for keywords such as "music", seeing that "electro-music" contains the word "music".

Entry = input("Enter topic: ")
Topic = Entry + "\n"
Readfilename = "Duplicates.txt"
Readfile = open(Readfilename, "r")
Readdata = Readfile.read()
Readfile.close()
if Topic not in Duplicates:
Filename = "Topics.txt"
File = open(Filename, "a")
File.append(Topic)
File.close()
Duplicate = Topic + "\n"
Readfile = open(Readfilename, "a")
Readfile.append(Topic)
Readfile.close()

Answer

You can read a file line by line which would result in a solution like this one

Entry = input("Enter topic: ")
Topic = Entry + "\n"
Readfilename = "Duplicates.txt"
found=False
with open(Readfilename, "r") as Readfile:
    for line in Readfile:
        if Topic==line:
            found=True
            break # no need to read more of the file

if not found:
    Filename = "Topics.txt"
    with open(Filename, "a") as File:
        File.write(Topic)

    with open(Readfilename, "a") as Readfile:
        Readfile.write(Topic)