Python Tips - Saving Time and Effort with CSVDictWriter and defaultdict

Posted on  Sep 2, 2016  in  Python Programming - Intermediate Level  by  Amo Chen  ‐ 1 min read

Python’s CSVDictWriter is a handy module that allows you to write dictionary data directly into a CSV file. Here’s how you can use it:

import csv

with open('my.csv', 'w') as csvf:
    writer = csv.DictWriter(csvf, fieldnames=[
        'field_1',
        'field_2',
        'field_3'
    ])

    writer.writeheader()
    writer.writerow({'field_1': '', 'field_2': 'b', 'field_3': ''})
    writer.writerow({'field_1': '', 'field_2': 'b', 'field_3': ''})

While CSVDictWriter is convenient, it stops if any fields in a row are missing. This can be a hassle if you have many fields, and not all of them have values. Do you really need to manually fill in every single field?

Actually, you can solve this problem by using defaultdict.

For example:

import csv
from collections import defaultdict

with open('my.csv', 'w') as csvf:
    writer = csv.DictWriter(csvf, fieldnames=[
        'field_1',
        'field_2',
        'field_3'
    ])

    writer.writeheader()
    for i in range(0, 10):
        row = defaultdict(str) # Create a dict with all keys defaulting to an empty string
        row['field_2'] = 'b'
        writer.writerow(row)

The idea is to use defaultdict to create a dict with default values. This way, when CSVDictWriter encounters fields that we haven’t set values for, defaultdict will automatically provide a default value for CSVDictWriter to use.

Simple, right?