DataLens

Reshaping, stacking, and splitting

Change an array's shape without changing its data, glue arrays together, and cut them into pieces.

Level: Intermediate · 9 min read

Practise this module

The questions for this lesson run in your browser and check your answer straight away.

The idea

Data rarely arrives in the shape you need. Twelve monthly numbers should be a 4 × 3 grid of quarters. A list of values must become a single column for scikit-learn. This month's data must be glued under last month's. NumPy has a small set of tools for this, and they all share one principle: the numbers stay the same, only how they're arranged changes.

reshape

Here's a year of monthly revenue as one long row of 12 numbers. The finance team wants it by quarter instead: four rows of three months each. The numbers don't change, only the layout, which is exactly what reshape does:

import numpy as np

monthly = np.array([335, 357, 388, 408, 410, 440, 452, 431, 470, 495, 520, 610])
quarters = monthly.reshape(4, 3)
print(quarters)
print(quarters.sum(axis=1))      # total per quarter
[[335 357 388]
 [408 410 440]
 [452 431 470]
 [495 520 610]]
[1080 1258 1353 1625]

reshape(4, 3) fills the new shape row by row: the first 3 values become row 0 (Q1), the next 3 row 1, and so on. The total number of elements must stay the same (4 × 3 = 12):

monthly.reshape(5, 3)
ValueError: cannot reshape array of size 12 into shape (5,3)

Let NumPy work out one dimension by writing -1:

print(monthly.reshape(-1, 3).shape)    # "3 columns, as many rows as needed"
print(monthly.reshape(2, -1).shape)    # "2 rows, as many columns as needed"
(4, 3)
(2, 6)

Rows to columns: the (n, 1) shape

Machine learning libraries expect the input X to be 2-D: one row per example, one column per feature. A single feature stored as a 1-D array must become a column:

ad_spend = np.array([5.0, 7.5, 6.0, 9.0])
print(ad_spend.shape)
X = ad_spend.reshape(-1, 1)
print(X.shape)
print(X)
(4,)
(4, 1)
[[5. ]
 [7.5]
 [6. ]
 [9. ]]

That's the fix for scikit-learn's famous error "Expected 2D array, got 1D array instead". Two other spellings do the same thing: ad_spend[:, np.newaxis] and np.expand_dims(ad_spend, axis=1).

Flatten: back to 1-D

The opposite is also common: a function or a chart wants one long list of values, and you have a grid. Flattening puts all the elements back into a single row:

print(quarters.ravel())
print(quarters.flatten())
[335 357 388 408 410 440 452 431 470 495 520 610]
[335 357 388 408 410 440 452 431 470 495 520 610]

Both return the elements in row order. ravel returns a view when it can (fast, shares data); flatten always copies (safe to modify).

Transpose: swap rows and columns

Sometimes data arrives the wrong way round: you wanted months as rows and quarters as columns, but got it the other way. Transposing flips the grid over its diagonal, so rows become columns and columns become rows:

print(quarters.T)
print(quarters.T.shape)
[[335 408 452 495]
 [357 410 431 520]
 [388 440 470 610]]
(3, 4)

.T turns the 4 quarters × 3 months grid into 3 × 4. You'll need it whenever data arrives the "wrong way round", and constantly in linear algebra.

Stacking: glue arrays together

Data often arrives in pieces. You might get January to March for two stores in one file and April to June in another, and need them as one six-month table. np.concatenate joins arrays along an axis they already have: axis=1 puts them side by side (more columns), and axis=0 puts them on top of each other (more rows):

h1 = np.array([[120, 135, 150], [80, 85, 90]])   # 2 stores × 3 months
h2 = np.array([[160, 155, 170], [88, 95, 100]])
print(np.concatenate([h1, h2], axis=1))   # side by side: more months
print(np.concatenate([h1, h2], axis=0))   # on top: more rows
[[120 135 150 160 155 170]
 [ 80  85  90  88  95 100]]
[[120 135 150]
 [ 80  85  90]
 [160 155 170]
 [ 88  95 100]]

The friendlier helpers are np.vstack (vertical: add rows) and np.hstack (horizontal: add columns):

new_store = np.array([40, 45, 43])
print(np.vstack([h1, new_store]))
[[120 135 150]
 [ 80  85  90]
 [ 40  45  43]]

np.column_stack builds a table from 1-D arrays used as columns, a common way to assemble features:

price = np.array([185, 320, 95])
cost = np.array([110, 210, 52])
table = np.column_stack([price, cost, price - cost])
print(table)
[[185 110  75]
 [320 210 110]
 [ 95  52  43]]

Shapes must agree on every axis except the one you're joining along, or NumPy raises an error naming the mismatch.

Splitting

The reverse of stacking. np.split cuts into equal pieces; np.array_split allows unequal ones:

first_half, second_half = np.split(monthly, 2)
print(first_half, second_half)

for part in np.array_split(np.arange(10), 3):
    print(part)
[335 357 388 408 410 440] [452 431 470 495 520 610]
[0 1 2 3]
[4 5 6]
[7 8 9]

You can also split at specific positions: np.split(monthly, [3, 6]) cuts before index 3 and before index 6, giving three pieces.

Why this matters for clean code

Reshaping is where shape bugs hide. A clean habit: write the expected shape in a comment or check it right after a reshape (assert X.shape == (n, 1)). And prefer reshape(-1, 1) over hard-coding the length, so the code keeps working when the data grows.

Recap

Practise this module

The questions for this lesson run in your browser and check your answer straight away.