Reading your data profile

Step 1 of the Dotwave pipeline — Understand your data — is where you build an honest picture of a dataset before you change a single value. It runs automatically the moment profiling finishes, and it summarizes everything Dotwave inferred about your file: how big it is, what each column contains, how complete it is, and where the obvious problems live. Reading this profile well is the difference between cleaning with intent and cleaning by guesswork. This article walks through every element on the screen and what it is telling you.

Reading the shape

The first thing to register is the dataset's shape — its size expressed as rows by columns. Rows are your records: one customer, one transaction, one survey response per row. Columns are the fields describing each record. A dataset that is 12,480 rows by 9 columns is a very different cleaning job from one that is 90 rows by 240 columns, and the shape tells you which you are dealing with before you scroll. A high column count relative to rows often signals a spreadsheet that has been pivoted or padded with derived fields; a very high row count tells you that any per-row operation you apply will run many times, so it is worth getting the logic right first.

The stat tiles

Beneath the shape, four stat tiles give you the health of the dataset at a glance. Read them together — each one frames the work in the next step:

• Rows — the total number of records in the file. This is your denominator for every percentage that follows.
• Columns — the total number of fields. Pair this with the shape to judge whether the table is wide or narrow.
• Duplicate rows — how many rows are exact copies of another row across every column. A non-zero figure here is your cue to inspect duplicates in Step 2 rather than assume they are errors.
• Total nulls — the count of missing cells summed across the entire dataset. A large number is not automatically alarming; it matters far more where those nulls are concentrated, which the column table shows you next.

The column info table

The column info table is the heart of the profile. It lists every column with three facts that drive most cleaning decisions:

• Dtype — the data type Dotwave inferred for the column, such as integer, float, text, or datetime. If a column you know to be a date shows as text, that mismatch is a cleaning task waiting for you in a later step.
• Non-null — how many rows actually contain a value in this column. Compare it against the total row count to see how complete the field is.
• Null count — how many rows are missing a value here. This is the per-column breakdown of the total-nulls tile, and it tells you exactly which columns need attention.

Scanning this table top to bottom gives you a prioritized worklist: the columns with the highest null counts and the most surprising dtypes are the ones to plan for first.

The data profile tabs and row distributions

Above the detail, the data profile is split into tabs so you can look at numeric and categorical columns separately, each with the statistics that suit its type. Switch between them to move from the numeric measures in your data to the text fields where inconsistent spellings hide. Within the table, you can expand any row to reveal that column's distribution chart — a small histogram for numeric columns or a frequency breakdown for categorical ones. Expanding a row is the fastest way to sanity-check a column: a numeric distribution with a long tail warns you about outliers, and a categorical breakdown with a dozen near-identical labels warns you about look-alike categories to merge later.

How Dotwave flags trouble

Dotwave colors the null figures so problems jump out without arithmetic. A column highlighted in amber is missing more than 10% of its values — enough to affect analysis, but usually fixable by filling. A column highlighted in red is missing more than 50% of its values, which is severe enough that filling may do more harm than good and dropping the column is often the better call. The color is a triage signal, not a command: red tells you to make a deliberate decision about that column rather than let a default fill quietly invent most of its data.