Performance — Profiling, Cython & Numba
Profile with cProfile and py-spy, optimise hot paths with Cython, JIT-compile with Numba, and benchmark with timeit.
Part 1: What You Will Learn
- Measure performance before attempting to optimise code.
- Profile functions with
cProfile. - Benchmark small operations with
timeit. - Recognise when Numba or Cython may be useful for CPU-heavy numerical code.
Part 2: Profile Before You Optimise
import cProfile
import pstats
from io import StringIO
def calculate_squares(limit: int) -> int:
total = 0
for number in range(limit):
total += number * number
return total
profiler = cProfile.Profile()
profiler.enable()
result = calculate_squares(1_000_000)
profiler.disable()
stream = StringIO()
stats = pstats.Stats(profiler, stream=stream)
stats.sort_stats("cumulative")
stats.print_stats(10)
print("Result:", result)
print(stream.getvalue())Part 3: Compare Implementations
from timeit import timeit
loop_code = '''
total = 0
for number in range(10000):
total += number * number
'''
sum_code = '''
total = sum(number * number for number in range(10000))
'''
print("Loop:", timeit(loop_code, number=1000))
print("sum :", timeit(sum_code, number=1000))timeit repeats short snippets many times to reduce timing noise. Always benchmark the real bottleneck rather than assuming a particular style is faster.
Part 4: Optional Numba Acceleration
from numba import njit
@njit
def calculate_squares(limit: int) -> int:
total = 0
for number in range(limit):
total += number * number
return total
print(calculate_squares(1_000_000))pip install numba
Numba can JIT-compile suitable numerical functions. Cython is another option when you need Python-like source compiled into a C extension. Both add complexity, so use them only after profiling shows that the CPU-heavy section matters.
Part 5: Hands-On Practice
Mini project — Performance Lab. Write two versions of a function that calculates the distance between many points. Profile both versions, record the execution time, then try a NumPy or Numba implementation. Explain which change produced the largest improvement and why.
Part 6: Next Steps
Keep a record of your before-and-after measurements, then continue to Lesson 36 to automate tests and deployment with CI/CD.