Learn About Map, Filter, Reduce, and Lambda with Python

2020-07-27 18:39:18 | #programming #python | Part 7 of 7

Tested On

  • Linux Ubuntu 20.04
  • Windows 10
  • macOS Catalina

Today, we're going to demonstrate some functional programming concepts by introducing you to map, filter, and reduce with Python.

As pure functions, they don't have side effects- which means they don't modify they state of parameters, but instead return new objects. And they iterate through recursion, meaning they repeatedly execute the same function/operation per element.

Python Map

The map function accepts two parameters: a function to perform on each element of an iterable, and an iterable. Because map is a pure function, it does not modify the iterable, but returns a brand new iterable with the modified values. And if you don't utilize break or continue, map will run on every single element, duplicating each of them, so you will need to be aware of the memory requirements of mapping a large collection. If it ends up being too memory-intensive, you may be better off with an old-fashioned for loop.

Try It Yourself


 
  

Python Filter

filter also accepts two parameters, but the difference is that the parameterized function you pass in must return a conditional True or False. Filter runs this function against each element, and if it evaluates to True, it stores element into a new iterable. Otherwise, it skips it. This new iterable ges returned at the end of the process. So for example, you can filter over a list of numbers and return a new list that contains only the even numbers:

Try It Yourself


 
  

Python Reduce

reduce also accepts a function and an iterable, but instead of returning a new iterable, it returns a single value that represents the cumulative effect of the function running on every single element. In Python 3.x, you need to import reduce by adding the following line to the top of your code: from functools import reduce

Try It Yourself


 
  

Python Lambda vs. Normal Functions

Lambda does not belong in the same pure function family as map, filter, and reduce. It's just a shorthand way of performing a function's operations without having to define the function with def.

Try It Yourself


 
  

Python Programming Exercises

Try to solve the following problems, using everything you've learned up to this point. Feel free to share solutions in the comments. Optimize each solution, as much as possible.

  1. Write a Python map function that accepts a dictionary and returns a JSON object. Run the map function against the following list of dictionaries and print the results to the console.

    Input:

    library = [{
        'title': 'Nineteen Eighty-Four',
        'author': 'George Orwell',
        'country': 'United Kingdom',
        'pages': 328,
        'genres': ['dystopian', 'political fiction'],
        'publisher': {
            'name': 'Secker & Warburg',
            'date': '1949-06-08 00:00:00'
        }
    }, {
        'title': 'Do Androids Dream of Electric Sheep?',
        'author': 'Phillip K. Dick',
        'country': 'United States',
        'pages': 210,
        'genres': ['science fiction', 'philosophical fiction'],
        'publisher': {'name': 'Doubleday', 'date': '1968-00-00 00:00:00'}
    }, {
        'title': 'Snow Crash',
        'author': 'William Gibson',
        'country': 'United States',
        'pages': 271,
        'genres': ['science fiction'],
        'publisher': {'name': 'Ace', 'date': '1984-07-01 00:00:00'}
    }]

    Expected Output: ['{"title": "Nineteen Eighty-Four", "author": "George Orwell", "country": "United Kingdom", "pages": 328, "genres": ["dystopian", "political fiction"], "publisher": {"name": "Secker & Warburg", "date": "1949-06-08 00:00:00"}}', '{"title": "Do Androids Dream of Electric Sheep?", "author": "Phillip K. Dick", "country": "United States", "pages": 210, "genres": ["science fiction", "philosophical fiction"], "publisher": {"name": "Doubleday", "date": "1968-00-00 00:00:00"}}', '{"title": "Snow Crash", "author": "William Gibson", "country": "United States", "pages": 271, "genres": ["science fiction"], "publisher": {"name": "Ace", "date": "1984-07-01 00:00:00"}}']

    import json
    
    
    def dict_to_json(book):
        return json.dumps(book)
    
    
    library = [{
        'title': 'Nineteen Eighty-Four',
        'author': 'George Orwell',
        'country': 'United Kingdom',
        'pages': 328,
        'genres': ['dystopian', 'political fiction'],
        'publisher': {
            'name': 'Secker & Warburg',
            'date': '1949-06-08 00:00:00'
        }
    }, {
        'title': 'Do Androids Dream of Electric Sheep?',
        'author': 'Phillip K. Dick',
        'country': 'United States',
        'pages': 210,
        'genres': ['science fiction', 'philosophical fiction'],
        'publisher': {'name': 'Doubleday', 'date': '1968-00-00 00:00:00'}
    }, {
        'title': 'Snow Crash',
        'author': 'William Gibson',
        'country': 'United States',
        'pages': 271,
        'genres': ['science fiction'],
        'publisher': {'name': 'Ace', 'date': '1984-07-01 00:00:00'}
    }]
    
    json_books = map(dict_to_json, library)
    print(list(json_books))
  2. Using the following csv file, write a Python map function that replaces all instances of "M" with "m" and all instances of "F" with "f" and saves the result to a new csv file

    Input:

    id,first_name,last_name,city,state,zip,dob,gender
    382,John,Doe,New York City,New York,10003,1959/05/23,M
    121,Jane,Doe,Paris,France,75000,1964/03/03,F
    329,Jack,Doe,Tokyo,Japan,100-0004,1961/07/15,M
    119,James,Doe,Ulaanbaatar,Mongolia,14200,1999/08/15,M
    124,Jamie,Doe,Casablanca,Morocco,20000,2000/12/10,F

    Expected Output:

    id,first_name,last_name,city,state,zip,dob,gender
    382,John,Doe,New York City,New York,10003,1959/05/23,m
    121,Jane,Doe,Paris,France,75000,1964/03/03,f
    329,Jack,Doe,Tokyo,Japan,100-0004,1961/07/15,m
    119,James,Doe,Ulaanbaatar,Mongolia,14200,1999/08/15,m
    124,Jamie,Doe,Casablanca,Morocco,20000,2000/12/10,f

    import csv
    
    
    def format_record(record):
        copy = record.copy()
        copy['gender'] = copy['gender'].lower()
        return copy
    
    
    with open('input.csv', 'r', newline='') as data:
        records = []
        rows = data.readlines()
        headers = [r.strip('\n\r') for r in rows[0].split(',')]
        for row in rows[1:]:
            record = {}
            for i, col in enumerate(row.split(',')):
                record[headers[i]] = col.rstrip('\n\r')
            records.append(record)
    
        with open('output.csv', mode='w') as csv_file:
            writer = csv.DictWriter(csv_file, fieldnames=records[0].keys(), delimiter=',')
    
            writer.writeheader()
            formatted = list(map(format_record, records))
            for record in formatted:
                writer.writerow(record)
  3. Write a Python filter function that keeps all users living in New York from the following list of dictionaries

    Input:

    records = [{
        'id': '1',
        'name': 'John Doe',
        'city': 'New York'
    }, {
        'id': '2',
        'name': 'Jane Doe',
        'city': 'New York'
    }, {
        'id': '3',
        'name': 'Jack Doe',
        'city': 'Florence'
    }, {
        'id': '4',
        'name': 'James Doe',
        'city': 'Hong Kong'
    }, {
        'id': '5',
        'name': 'Jamie Doe',
        'city': 'Sydney'
    }]

    Expected Output: [{'id': '1', 'name': 'John Doe', 'city': 'New York'}, {'id': '2', 'name': 'Jane Doe', 'city': 'New York'}]

    def is_ny_resident(record):
        return record['city'] == 'New York'
    
    
    records = [{
        'id': '1',
        'name': 'John Doe',
        'city': 'New York'
    }, {
        'id': '2',
        'name': 'Jane Doe',
        'city': 'New York'
    }, {
        'id': '3',
        'name': 'Jack Doe',
        'city': 'Florence'
    }, {
        'id': '4',
        'name': 'James Doe',
        'city': 'Hong Kong'
    }, {
        'id': '5',
        'name': 'Jamie Doe',
        'city': 'Sydney'
    }]
    
    filtered = list(filter(is_ny_resident, records))
    print(filtered)
  4. Write a Python filter function that removes books published before 1969 from the following list

    Input:

    library = [{
        'title': 'Nineteen Eighty-Four',
        'author': 'George Orwell',
        'country': 'United Kingdom',
        'pages': 328,
        'genres': ['dystopian', 'political fiction'],
        'publisher': {
            'name': 'Secker & Warburg',
            'date': '1949-06-08 00:00:00'
        }
    }, {
        'title': 'Do Androids Dream of Electric Sheep?',
        'author': 'Phillip K. Dick',
        'country': 'United States',
        'pages': 210,
        'genres': ['science fiction', 'philosophical fiction'],
        'publisher': {'name': 'Doubleday', 'date': '1968-01-01 00:00:00'}
    }, {
        'title': 'Snow Crash',
        'author': 'William Gibson',
        'country': 'United States',
        'pages': 271,
        'genres': ['science fiction'],
        'publisher': {'name': 'Ace', 'date': '1984-07-01 00:00:00'}
    }]

    Expected Output: [{'title': 'Nineteen Eighty-Four', 'author': 'George Orwell', 'country': 'United Kingdom', 'pages': 328, 'genres': ['dystopian', 'political fiction'], 'publisher': {'name': 'Secker & Warburg', 'date': '1949-06-08 00:00:00'}}, {'title': 'Do Androids Dream of Electric Sheep?', 'author': 'Phillip K. Dick', 'country': 'United States', 'pages': 210, 'genres': ['science fiction', 'philosophical fiction'], 'publisher': {'name': 'Doubleday', 'date': '1968-01-01 00:00:00'}}]

    from datetime import datetime
    
    
    def is_pre_1969(book):
        d = datetime.strptime(book['publisher']['date'], '%Y-%m-%d %H:%M:%S')
        return d.year < 1969
    
    
    library = [{
        'title': 'Nineteen Eighty-Four',
        'author': 'George Orwell',
        'country': 'United Kingdom',
        'pages': 328,
        'genres': ['dystopian', 'political fiction'],
        'publisher': {
            'name': 'Secker & Warburg',
            'date': '1949-06-08 00:00:00'
        }
    }, {
        'title': 'Do Androids Dream of Electric Sheep?',
        'author': 'Phillip K. Dick',
        'country': 'United States',
        'pages': 210,
        'genres': ['science fiction', 'philosophical fiction'],
        'publisher': {'name': 'Doubleday', 'date': '1968-01-01 00:00:00'}
    }, {
        'title': 'Snow Crash',
        'author': 'William Gibson',
        'country': 'United States',
        'pages': 271,
        'genres': ['science fiction'],
        'publisher': {'name': 'Ace', 'date': '1984-07-01 00:00:00'}
    }]
    
    filtered = list(filter(is_pre_1969, library))
    print(filtered)
  5. Write a Python filter function that keeps all users between the ages of 18-35 from the following csv, and save the result to a new csv file

    Input:

    id,first_name,last_name,city,state,zip,age,gender
    382,John,Doe,New York City,New York,10003,54,M
    121,Jane,Doe,Paris,France,75000,48,F
    329,Jack,Doe,Tokyo,Japan,100-0004,10,M
    119,James,Doe,Ulaanbaatar,Mongolia,14200,25,M
    124,Jamie,Doe,Casablanca,Morocco,20000,20,F

    Expected Output:

    id,first_name,last_name,city,state,zip,age,gender
    id,first_name,last_name,city,state,zip,age,gender
    119,James,Doe,Ulaanbaatar,Mongolia,14200,25,M
    124,Jamie,Doe,Casablanca,Morocco,20000,20,F

    import csv
    
    
    def is_demographic(record):
        age = int(record['age'])
        return age >= 18 and age <= 35
    
    
    with open('input.csv', 'r', newline='') as data:
        records = []
        rows = data.readlines()
        headers = [r.strip('\n\r') for r in rows[0].split(',')]
        for row in rows[1:]:
            record = {}
            for i, col in enumerate(row.split(',')):
                record[headers[i]] = col.rstrip('\n\r')
            records.append(record)
    
        with open('output.csv', mode='w') as csv_file:
            writer = csv.DictWriter(csv_file, fieldnames=records[0].keys(), delimiter=',')
    
            writer.writeheader()
            formatted = list(filter(is_demographic, records))
            for record in formatted:
                writer.writerow(record)

Want To See More Exercises?

View Exercises

Comments

You must log in to comment. Don't have an account? Sign up for free.

Subscribe to comments for this post

Want To Receive More Free Content?

Would you like to receive free resources, tailored to help you reach your IT goals? Get started now, by leaving your email address below. We promise not to spam. You can also sign up for a free account and follow us on and engage with the community. You may opt out at any time.



Tell Us About Your Project









Contact Us

Do you have a specific IT problem that needs solving or just have a general IT question? Use the contact form to get in touch with us and an IT professional will be with you, momentarily.

Hire Us

We offer web development, enterprise software development, QA &amp; testing, google analytics, domains and hosting, databases, security, IT consulting, and other IT-related services.

Free IT Tutorials

Head over to our tutorials section to learn all about working with various IT solutions.

Contact