Notebooks on Ye Joo Park's Blog

Using GPT to Generate Paper Feedback

Mon, 08 Sep 2025 00:00:00 +0000

In 2012, I graded hundreds of papers as a teaching assistant. This was a time-consuming and subjective process, and I often spent days grading just to meet deadlines.

Large language models (LLMs) like GPT-5 are transforming education, including the grading of written assignments. LLMs provide an efficient and scalable way to assess students' writing while also generating personalized feedback. However, my experience using LLMs for grading has been unreliable for the following reasons:

Analyzing College Town Landlords at UIUC, BYU, and PSU

Thu, 30 Jan 2025 00:00:00 +0000

✨ Background¶

I first moved into an apartment at UIUC during the summer of 2007. The apartment was Gregory Place under a management company called JSM. Here is a photo of the apartment from 2007. The construction site shows the second building being built.

Back in 2007, JSM was regarded as one of the better management companies, along with Royse Brinkmeyer and Roland. Almost two decades later, JSM is still highly regraded. Prospective tenants lining up from the early morning in front of JSM's office to secure a lease was a sight to see in recent years.

Sentiment Analysis using spaCy and DistilBERT

Thu, 23 Jan 2025 00:00:00 +0000

Natural language processing (NLP) aims to give computers the ability to understand, process, and even generate human language. This notebook introduces the common preprocessing steps and demonstrates how to use a widely used transformer model (distilbert-base-uncased-finetuned-sst-2-english) to perfrom a sentiment analysis. 😀😦🙁

In [1]:

import pandas as pd
import numpy as np
import plotly.express as px

In [2]:

pd.set_option('display.max_columns', 100)

🗃️ Load data¶

This exercise uses a small dataset that contains reviews of two apartments at Indiana University Bloomington.

Evaluating LLMs' DataViz Code Generation and Refinement Capabilities

Sun, 29 Sep 2024 00:00:00 +0000

Refining a visual often involves getting rid of unnecessary elements and directing a viewer's attention to specific elements. The process to enhance clearity, readability, and effectiveness is a iterative process that requires constant self-evaluations. This process requires many hours, although the output may look simple.

Can Large Language Models (LLMs) be used as a tool for generating visuals? While LLMs excel at text-based tasks, their ability to understand and generate complex concepts can be leveraged to assist in writing code to create visuals. But can LLMs be used to refine visuals? This post tests LLM's ability to refine Plotly visuals.

Benford's Law Application and Interpretation

Sat, 10 Feb 2024 00:00:00 +0000

Benford's Law, also known as the Newcomb-Benford law or the first-digit law, is a surprising observation about the leading digits of numbers in real-world datasets. In many naturally occurring collections of data, smaller leading digits (like 1 and 2) are significantly more common than larger ones (like 8 and 9).

Financial records
Scientific measurements
Astronomical distances
Street addresses

Why does this happen?¶

Real-world data often involves growth, multiplication, and comparisons across different scales. This "scaling invariance" creates a natural bias towards smaller leading digits.

An In-depth Comparison of Pandas String dtypes

Sat, 27 May 2023 00:00:00 +0000

In [1]:

import sys
import pandas as pd
import numpy as np

print(f"python version {sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}")
print(f"pandas version {pd.__version__}")
print(f"numpy version {np.__version__}")

python version 3.10.11
pandas version 1.5.2
numpy version 1.23.5

The default data type for strings in Pandas DataFrames is the object type. However, pandas' documentation recommendeds explicitly using the StringDtype for storing strings as it's more efficient and allows for more specific string operations.

pd.StringDtype() is a dedicated data type for storing strings. It is an It allows for more specific string operations. StringDtype is still considered experimental as of pandas 2.0.1.

Retrieve tweets using Twitter API

Wed, 08 Feb 2023 00:00:00 +0000

A Twitter Developer account is required to run this script.

In [1]:

import tweepy
import json
import pandas as pd
import numpy as np

In [2]:

pd.set_option('display.max_colwidth', 150)
pd.set_option('display.max_rows', 20)

Read Twitter API credentials¶

Read Twitter API credentials from twitter-credentials.json. The JSON file should contain the following key/values:

{
 "consumer_key": "YOUR_CONSUMER_KEY",
 "consumer_secret": "YOUR_CONSUMER_SECRET",
 "access_token": "YOUR_ACCESS_TOKEN",
 "access_token_secret": "YOUR_ACCESS_TOKEN_SECRET",
 "bearer_token": "BEARER_TOKEN"
}

Create the twitter-credentials.json with your own keys and tokens in the same folder as this Jupyter notebook.

Cleaning Chicago Ridesharing Registered Vehicles Dataset

Fri, 12 Mar 2021 00:00:00 +0000

All vehicles 🚗 reported by Transportation Network Providers (sometimes called rideshare companies) to the City of Chicago as part of routine reporting required by ordinance. Inclusion of a vehicle in a monthly report indicates that the vehicle was eligible for trips in Chicago in that month for at least one day, regardless of whether it actually provided any rides. If a vehicle is eligible in multiple months, which is common, it will have records in each of these reporting months.

COVID-19 Exploratory Data Analysis

Mon, 01 Jun 2020 00:00:00 +0000

This is a notebook from the live coding session by Dr Hugo Bowne-Anderson on April 10, 2020 via DataCamp.

Imports and data¶

Let's import the necessary packages from the SciPy stack and get the data.

In [1]:

# Import packages
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
# Set style & figures inline
sns.set()
%matplotlib inline

In [2]:

# Data urls
base_url = 'https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_time_series/'
confirmed_cases_data_url = base_url + 'time_series_covid19_confirmed_global.csv'
death_cases_data_url = base_url + 'time_series_covid19_deaths_global.csv'
recovery_cases_data_url = base_url+ 'time_series_covid19_recovered_global.csv'
# Import datasets as pandas dataframes
raw_data_confirmed = pd.read_csv(confirmed_cases_data_url)
raw_data_deaths = pd.read_csv(death_cases_data_url)
raw_data_recovered = pd.read_csv(recovery_cases_data_url)

Confirmed cases of COVID-19¶

We'll first check out the confirmed cases data by looking at the head of the dataframe: