Tag
Messy data slows down every analysis that follows. These prompts help you spot inconsistent formats, duplicate rows, and outliers before they skew your results.
7 prompts
Audits a dataset's health before analysis starts, using its columns, row count, source, time range, and a sample of rows to spot missing values, type mismatches, likely outliers, duplicate rows, and inconsistent categorical entries. Returns a 1-10 quality score with reasoning and a concrete cleaning step for every issue found, making it a natural first pass before writing any cleaning script.
Builds a repeatable validation suite for a dataset you'll keep receiving, based on its description, columns, and stated business rules. Produces per-column checks, cross-column business-logic rules, and statistical sanity checks, then wraps them into a Python script that reports failure counts per rule, sample bad rows, an overall quality score, and a priority order for fixes.
Handles the type-conversion and encoding grunt work needed before a dataset is ready for analysis or modeling. - Input: a list of columns with their current and target types - Converts types with edge-case handling (stray text in numeric columns, mixed-type columns) - Encodes categoricals appropriately: ordinal, one-hot/target, or frequency encoding for high-cardinality fields - Output: Python code plus a before/after conversion report flagging failed conversions
Designs a deduplication pipeline for records — customers, transactions, or products — that carry small formatting or spelling variations instead of exact duplicates, using the dataset description, key identifying fields, and known variation types as inputs. Combines exact-match removal with fuzzy techniques like Levenshtein or phonetic matching, assigns a confidence score to each match, proposes a merge strategy, and generates a human-reviewable report before anything is deleted.
List the columns with missing data — name, type, approximate percent missing, and likely cause — and this prompt classifies each one as MCAR, MAR, or MNAR before recommending a fix. It weighs whether the missingness itself is informative, considers adding a was_missing flag, justifies the chosen imputation method against alternatives for every column, and outputs the Python code implementing the strategy column by column.
Give the numeric columns to check and describe what the dataset represents, and this prompt detects outliers with IQR, Z-score, and isolation forest in parallel, then classifies each one as a data-entry error, a genuine extreme value, or part of a systemic cluster pointing to an upstream process bug. It shows each outlier alongside its row context, quantifies how much removing them shifts the mean and median, and generates Python code that flags the outliers and outputs a cleaned dataset with the recommended handling applied.
List the current messy column names, the target clean names, and your formatting rules for dates, phone numbers, and category mappings. - Generates a runnable pandas script standardizing dates and currency - Splits addresses into street, city, state, and zip - Splits full names into first_name and last_name, formats phone numbers - Ends with a before/after validation summary of row counts and samples