In today’s digital landscape, understanding public interests through search trends offers invaluable insights into market dynamics and public opinion. Google Trends provides a straightforward way to track search term popularity across different regions, benefiting everyone from marketers to casual observers. However, for deeper analysis, this interface may not suffice. This is where Python comes into play, offering the tools to delve deeper and automate the extraction process, allowing for a more comprehensive understanding of the data.
In this tutorial, we will explore how to extract Google Trends data using Python, making the most of the pytrends library to programmatically access this rich source of information. By integrating Python with this powerful tool, you can customize your data extraction, handle larger datasets, and even combine Google Trends data with other sources for more comprehensive insights.
Additionally, we’ll utilize tkinter to create a user-friendly graphical interface, making it easier and more intuitive to input your data queries and visualize the results. This integration not only enhances functionality but also makes your data analysis tasks more accessible.
Let’s get started!
Table of Contents
- Necessary Libraries
- Imports
- Initializing Google Trends Request Object
- Creating GUI Widgets
- Handle Request Rate Limiting
- Defining Functions for API Requests
- Plotting and Display Function
- Display Data in a New Window
- Main Execution Block
- Example
- Full Code
Necessary Libraries
Make sure to install these libraries via the terminal or your command prompt for the code to function properly:
$ pip install tk
$ pip install pytrends
$ pip install seaborn
$ pip install matplotlib
$ pip install pandas
Imports
from pytrends.request import TrendReq
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
from tkinter import *
from tkinter import messagebox
from tkinter import ttk
import time
Imagine you’re on a quest to uncover the world’s collective curiosity, to discover what people are searching for. Before embarking on this adventure, you need to gear up with the right tools. Let’s look at our imports:
- TrendReq from pytrends: This serves as our gateway to the Google Trends API, allowing us to make requests and fetch data.
- seaborn: Used for creating visually appealing statistical graphics.
- matplotlib.pyplot: Enables us to create a variety of static, interactive, and animated visualizations.
- pandas: A powerful tool for data manipulation and analysis.
- tkinter, along with messagebox and ttk: These are used to build a graphical user interface that’s both functional and appealing, including message boxes for alerts and themed widgets for a polished look.
- time: Important for handling time-related functions, like pauses or delays in processing.
After loading these libraries, we set the style of our seaborn plots to “whitegrid“, which offers a clean and structured background for our charts.
# Set the styling for the plots
sns.set_style("whitegrid")
Initializing Google Trends Request Object
Now, since we need to interact with Google Trends to learn about global curiosities, it’s essential that we ‘speak the same language‘. The initialize_trends() function helps us do just that by setting the language to US English and aligning the timestamp to UTC+6.
def initialize_trends():
"""Initialize the Google Trends request object."""
return TrendReq(hl="en-US", tz=360)
Creating GUI Widgets
This section is where we build our control panel, which includes buttons, entry fields, and labels—essentially, everything we need to command our data extraction. With that in mind, let’s delve into the details of the create_widgets() function:
- This function sets up labels that prompt the user for inputs, including a label that serves as the title of this application. It also creates buttons that, when clicked, trigger specific functions. For example, the “Get Related Topics” button activates the
get_related_topics()function, and so forth.
def create_widgets(root, pytrends):
Label(root, text="Google Trends Data Extraction", font=("Helvetica", 16)).grid(row=0, column=0, columnspan=2,
pady=10)
Label(root, text="Enter Keywords (comma-separated):").grid(row=1, column=0, padx=10, pady=5, sticky=E)
keywords_entry = Entry(root, width=50)
keywords_entry.grid(row=1, column=1, padx=10, pady=5)
Label(root, text="Enter Timeframe (default is 'all'):").grid(row=2, column=0, padx=10, pady=5, sticky=E)
timeframe_entry = Entry(root, width=50)
timeframe_entry.grid(row=2, column=1, padx=10, pady=5)
Button(root, text="Get Interest Over Time",
command=lambda: get_interest_over_time(pytrends, keywords_entry, timeframe_entry)).grid(row=3, column=0,
columnspan=2,
pady=10)
Button(root, text="Get Interest by Region", command=lambda: get_interest_by_region(pytrends, keywords_entry)).grid(
row=4, column=0, columnspan=2, pady=10)
Button(root, text="Get Related Topics", command=lambda: get_related_topics(pytrends, keywords_entry)).grid(row=5,
column=0,
columnspan=2,
pady=10)
Button(root, text="Get Related Queries", command=lambda: get_related_queries(pytrends, keywords_entry)).grid(row=6,
column=0,
columnspan=2,
pady=10)
Button(root, text="Get Suggestions", command=lambda: get_suggestions(pytrends, keywords_entry)).grid(row=7,
column=0,
columnspan=2,
pady=10)
Button(root, text="Get Trending Searches", command=lambda: get_trending_searches(pytrends)).grid(row=8, column=0,
columnspan=2,
pady=10)
Button(root, text="Get Real-Time Trending Searches", command=lambda: get_realtime_trending_searches(pytrends)).grid(
row=9, column=0, columnspan=2, pady=10)
Handle Request Rate Limiting
Every journey comes with its obstacles, and our data extraction journey is no exception. To ensure we can continue our adventure despite any setbacks, we’ve implemented the handle_request() function. This function is designed to manage Google Trends API rate limiting errors. How does it work?
- First, it wraps the decorated function inside a
wrapper()function. When the decorated function runs, thewrapper()handles any errors. If an exception occurs, it checks if the error message includes ‘429’, indicating the rate limit was exceeded. If so, it shows an informational message, waits 60 seconds, and retries the function. If the function fails again, it displays an error message explaining the issue.
def handle_request(func):
"""Decorator to handle request rate limiting."""
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
except Exception as e:
if "429" in str(e):
messagebox.showinfo("Rate Limit Exceeded", "Rate limit exceeded. Waiting for 60 seconds.")
time.sleep(60)
return func(*args, **kwargs)
else:
messagebox.showerror("Error", str(e))
return wrapper
Defining Functions for API Requests
Now, with the safety net in place, we can proceed without worry, knowing that any rate-limiting issues will be handled gracefully. Let’s define our functions for API Requests:
Interest Over Time
With the get_interest_over_time() function, we can track how interest in the entered keywords evolves over a specified timeframe. This function is triggered when a button created in create_widgets() is clicked. It retrieves the keywords from the entry field, splits them into a list, and checks if a timeframe was provided; if not, it defaults to ‘all’. Then, it uses pytrends to build the payload with these inputs, fetches the Interest Over Time data, and calls the plot_interest_over_time() function to visualize the data.
@handle_request
def get_interest_over_time(pytrends, keywords_entry, timeframe_entry):
keywords = keywords_entry.get().split(',')
timeframe = timeframe_entry.get() or "all"
pytrends.build_payload(keywords, timeframe=timeframe)
interest_over_time_df = pytrends.interest_over_time()
plot_interest_over_time(interest_over_time_df, "Interest Over Time")
Interest by Region
The purpose of the get_interest_by_region() function is to see how interest in the chosen keywords varies by region. This function is triggered by a button created in create_widgets(). It retrieves the first keyword from the entry field, uses pytrends to build the payload with this keyword, and fetches the Interest by Region data. The data is then sorted by interest in descending order and displayed using the display_data() function.
@handle_request
def get_interest_by_region(pytrends, keywords_entry):
keyword = keywords_entry.get().split(',')[0]
pytrends.build_payload([keyword], timeframe="all")
region_interest = pytrends.interest_by_region(resolution="COUNTRY", inc_low_vol=True, inc_geo_code=True)
sorted_regions = region_interest[keyword].sort_values(ascending=False).reset_index()
display_data(sorted_regions)
Related Topics
The get_related_topics() function aims to fetch topics related to the first keyword. Triggered by a button click from the create_widgets() function, it follows a process similar to the previous function. The main difference is that it uses pytrends to retrieve related topics for the keyword and then displays this information using the display_data() function.
@handle_request
def get_related_topics(pytrends, keywords_entry):
keyword = keywords_entry.get().split(',')[0]
pytrends.build_payload([keyword], timeframe="all")
related_topics = pytrends.related_topics()
display_data(related_topics[keyword]["top"].reset_index())
Related Queries
This one works in the same way as the previous function. However, this one uses pytrends to retrieve queries related to the keyword and displays them using the display_data() function.
@handle_request
def get_related_queries(pytrends, keywords_entry):
keyword = keywords_entry.get().split(',')[0]
pytrends.build_payload([keyword], timeframe="all")
related_queries = pytrends.related_queries()
display_data(related_queries[keyword]["top"].reset_index())
Suggestions
Now it’s time for us to get suggestions for our first keyword using the get_suggestions() function. How does it work? First, it retrieves the keyword, then it uses pytrends to get suggestions data, converts them into a DataFrame using pandas, and finally displays them with the display_data() function. Like the previous four functions, this one is also triggered by a button click created in create_widgets().
@handle_request
def get_suggestions(pytrends, keywords_entry):
keyword = keywords_entry.get().split(',')[0]
suggestions = pytrends.suggestions(keyword)
display_data(pd.DataFrame(suggestions))
Trending Searches
When this function is triggered by a button click created in create_widgets(), it uses pytrends to retrieve the trending searches for the specified region—in this case, the United Kingdom. You can change the region to, for example: the United States. The results are then displayed using the display_data() function.
@handle_request
def get_trending_searches(pytrends):
trending_searches = pytrends.trending_searches(pn="united_kingdom")
display_data(trending_searches.reset_index())
Real-Time Trending Searches
The last function for API requests works similarly to the previous function, but with one twist: it fetches real-time trending searches, as its name suggests, and displays them using the display_data() function.
@handle_request
def get_realtime_trending_searches(pytrends):
realtime_trending_searches = pytrends.realtime_trending_searches()
display_data(realtime_trending_searches.reset_index())
Plotting and Display Function
Basically, the plot_interest_over_time() function transforms raw data into a visual story. It uses pandas to plot the interest over time data and then displays it with the plt.show() function.
def plot_interest_over_time(data, title):
ax = data.plot(figsize=(12, 8), title=title)
ax.set_xlabel("Date")
ax.set_ylabel("Interest")
plt.legend(title="Keywords")
plt.show()
Display Data in a New Window
For this step, we create the display_data() function to open a window that displays the results of the functions that call it. This window has a title and defined geometry. We also create a frame to contain its elements, such as the treeview widget and the scrollbar.
Additionally, we use the grid layout to ensure the output displays in a neatly organized table.
def display_data(data):
top = Toplevel()
top.title("Data Display - The Pycodes")
# Create a treeview with vertical and horizontal scrollbars
frame = Frame(top)
frame.pack(fill=BOTH, expand=1)
tree = ttk.Treeview(frame, columns=list(data.columns), show="headings")
tree.pack(side=LEFT, fill=BOTH, expand=1)
vsb = ttk.Scrollbar(frame, orient="vertical", command=tree.yview)
vsb.pack(side=RIGHT, fill=Y)
hsb = ttk.Scrollbar(frame, orient="horizontal", command=tree.xview)
hsb.pack(side=BOTTOM, fill=X)
tree.configure(yscrollcommand=vsb.set, xscrollcommand=hsb.set)
for col in data.columns:
tree.heading(col, text=col)
tree.column(col, width=100)
for index, row in data.iterrows():
tree.insert("", "end", values=list(row))
top.geometry("600x400")
top.mainloop()
Main Execution Block
This is the grand finale, the block that ensures the script runs directly and not as an imported module. Here, we initialize our main window, set its title, initialize the Google Trends request object, and call the create_widgets() function to create the GUI elements on the main window.
Finally, we start the main event loop so the main window keeps running and remains responsive to the user.
if __name__ == "__main__":
root = Tk()
root.title("Google Trends Data Extraction App - The Pycodes")
pytrends = initialize_trends()
create_widgets(root, pytrends)
root.mainloop()
Example
I ran this code on a Windows system as shown in the images below:
I entered ‘Python’ as a keyword and set the timeframe from the first day of last year until today:
And for Linux users, this script also works just fine as shown in the image below:
Full Code
from pytrends.request import TrendReq
import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd
from tkinter import *
from tkinter import messagebox
from tkinter import ttk
import time
# Set the styling for the plots
sns.set_style("whitegrid")
def initialize_trends():
"""Initialize the Google Trends request object."""
return TrendReq(hl="en-US", tz=360)
def create_widgets(root, pytrends):
Label(root, text="Google Trends Data Extraction", font=("Helvetica", 16)).grid(row=0, column=0, columnspan=2,
pady=10)
Label(root, text="Enter Keywords (comma-separated):").grid(row=1, column=0, padx=10, pady=5, sticky=E)
keywords_entry = Entry(root, width=50)
keywords_entry.grid(row=1, column=1, padx=10, pady=5)
Label(root, text="Enter Timeframe (default is 'all'):").grid(row=2, column=0, padx=10, pady=5, sticky=E)
timeframe_entry = Entry(root, width=50)
timeframe_entry.grid(row=2, column=1, padx=10, pady=5)
Button(root, text="Get Interest Over Time",
command=lambda: get_interest_over_time(pytrends, keywords_entry, timeframe_entry)).grid(row=3, column=0,
columnspan=2,
pady=10)
Button(root, text="Get Interest by Region", command=lambda: get_interest_by_region(pytrends, keywords_entry)).grid(
row=4, column=0, columnspan=2, pady=10)
Button(root, text="Get Related Topics", command=lambda: get_related_topics(pytrends, keywords_entry)).grid(row=5,
column=0,
columnspan=2,
pady=10)
Button(root, text="Get Related Queries", command=lambda: get_related_queries(pytrends, keywords_entry)).grid(row=6,
column=0,
columnspan=2,
pady=10)
Button(root, text="Get Suggestions", command=lambda: get_suggestions(pytrends, keywords_entry)).grid(row=7,
column=0,
columnspan=2,
pady=10)
Button(root, text="Get Trending Searches", command=lambda: get_trending_searches(pytrends)).grid(row=8, column=0,
columnspan=2,
pady=10)
Button(root, text="Get Real-Time Trending Searches", command=lambda: get_realtime_trending_searches(pytrends)).grid(
row=9, column=0, columnspan=2, pady=10)
def handle_request(func):
"""Decorator to handle request rate limiting."""
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
except Exception as e:
if "429" in str(e):
messagebox.showinfo("Rate Limit Exceeded", "Rate limit exceeded. Waiting for 60 seconds.")
time.sleep(60)
return func(*args, **kwargs)
else:
messagebox.showerror("Error", str(e))
return wrapper
@handle_request
def get_interest_over_time(pytrends, keywords_entry, timeframe_entry):
keywords = keywords_entry.get().split(',')
timeframe = timeframe_entry.get() or "all"
pytrends.build_payload(keywords, timeframe=timeframe)
interest_over_time_df = pytrends.interest_over_time()
plot_interest_over_time(interest_over_time_df, "Interest Over Time")
@handle_request
def get_interest_by_region(pytrends, keywords_entry):
keyword = keywords_entry.get().split(',')[0]
pytrends.build_payload([keyword], timeframe="all")
region_interest = pytrends.interest_by_region(resolution="COUNTRY", inc_low_vol=True, inc_geo_code=True)
sorted_regions = region_interest[keyword].sort_values(ascending=False).reset_index()
display_data(sorted_regions)
@handle_request
def get_related_topics(pytrends, keywords_entry):
keyword = keywords_entry.get().split(',')[0]
pytrends.build_payload([keyword], timeframe="all")
related_topics = pytrends.related_topics()
display_data(related_topics[keyword]["top"].reset_index())
@handle_request
def get_related_queries(pytrends, keywords_entry):
keyword = keywords_entry.get().split(',')[0]
pytrends.build_payload([keyword], timeframe="all")
related_queries = pytrends.related_queries()
display_data(related_queries[keyword]["top"].reset_index())
@handle_request
def get_suggestions(pytrends, keywords_entry):
keyword = keywords_entry.get().split(',')[0]
suggestions = pytrends.suggestions(keyword)
display_data(pd.DataFrame(suggestions))
@handle_request
def get_trending_searches(pytrends):
trending_searches = pytrends.trending_searches(pn="united_kingdom")
display_data(trending_searches.reset_index())
@handle_request
def get_realtime_trending_searches(pytrends):
realtime_trending_searches = pytrends.realtime_trending_searches()
display_data(realtime_trending_searches.reset_index())
def plot_interest_over_time(data, title):
ax = data.plot(figsize=(12, 8), title=title)
ax.set_xlabel("Date")
ax.set_ylabel("Interest")
plt.legend(title="Keywords")
plt.show()
def display_data(data):
top = Toplevel()
top.title("Data Display - The Pycodes")
# Create a treeview with vertical and horizontal scrollbars
frame = Frame(top)
frame.pack(fill=BOTH, expand=1)
tree = ttk.Treeview(frame, columns=list(data.columns), show="headings")
tree.pack(side=LEFT, fill=BOTH, expand=1)
vsb = ttk.Scrollbar(frame, orient="vertical", command=tree.yview)
vsb.pack(side=RIGHT, fill=Y)
hsb = ttk.Scrollbar(frame, orient="horizontal", command=tree.xview)
hsb.pack(side=BOTTOM, fill=X)
tree.configure(yscrollcommand=vsb.set, xscrollcommand=hsb.set)
for col in data.columns:
tree.heading(col, text=col)
tree.column(col, width=100)
for index, row in data.iterrows():
tree.insert("", "end", values=list(row))
top.geometry("600x400")
top.mainloop()
if __name__ == "__main__":
root = Tk()
root.title("Google Trends Data Extraction App - The Pycodes")
pytrends = initialize_trends()
create_widgets(root, pytrends)
root.mainloop()
Happy Coding!
