Published 2020-10-22.
Time to read: 3 minutes.

This page is part of the posts collection, categorized under Python, Scala.

Pipes are the ultimate in functional programming. It is clear when reading code that uses pipes that data is not mutated. Piping data into and out of lambda functions (and regular functions) is a succinct way of elegantly expressing a computation. This article discusses how to do this using similar syntax in Python 3, Scala 2 and Scala 3.

After programming in Scala for more than ten years I have grown to appreciate Scala's implementation of lambda functions, including the ability to use the underscore character as a placeholder for variables. Scala 2.13 introduced pipelining between functions, which is rather like *nix pipes between processes.

Python 3 can also do something similar. This article demonstrates how to use sspipe and JulienPalard’s pipe with Scala's underscore placeholder for Python 3 lambda functions.

Python 3 Setup

The key concept is to use this specific Python import:

Shell

from sspipe import p, px, px as _

All the Python code examples that follow require this import. The Python code examples are modified versions of the sspipe examples to illustrate how to use underscores as placeholders.

This import is unusual because it imports px twice: once as a normal import, and once aliased to _. I use the _ alias to support Scala-like syntax, and I use px when I need to reference a parameter twice. Python is unlike Scala in that the Python compiler does not treat variables called _ specially; those variables are merely called _. I could use _ in Python code many times to refer to the same value, but a Scala programmer reading that code would expect that each reference to _ would be another input parameter, not a regular variable reference. The examples that follow should make this clear.

Python 3 Installation

Install sspipe using pip:

Shell

$ pip install --upgrade sspipe

Scala 2.13+ Setup

Scala 2.13 introduced ChainingOps, which adds chaining methods tap and pipe to every type. The key concept is to import the following prior to attempting the code examples below:

Shell

implicit class Smokin[A](val a: A) {
    import scala.util.chaining._
    import scala.language.implicitConversions

    implicit def |>[B](f: (A) => B): B = a.pipe(f)
}

Python and Scala Usage Examples

One Lambda Function and 1 Pipe

This Python example employs one lambda function and 1 pipe to add 2 to the number 5:

Shell

>>> 5 | _ + 2
7 

The Scala equivalent of the above is:

Shell

scala> 5 |> ((_: Int) + 2)
val res0: Int = 7 

Two Lambda Functions and 2 Pipes

This Python example employs two lambda functions and 2 pipes to multiply the previous result by 5 and then add the previous result. Recall that I said that in Python, an underscore when used this way is the name of a normal variable and that the compiler does not treat underscores as placeholders for lambda parameters. A Scala programmer would complain about the following code, because they would expect that the second lambda function would require 2 inputs:

Shell

>>> 5 | _ + 2 | _ * 5 + _
42 

A better way to write the above would be to use the special variable px, which was imported above. Now everyone either knows that px holds the piped value, or they complain about px being a magic variable. A possible solution to this complaint would be to alias px to a more descriptive name, such as pipedValue … which is still magical, but at least it is more descriptive.

Shell

>>> 5 | _ + 2 | px * 5 + px
42 

The Scala equivalent of the above is:

Shell

scala> 5 |> ((_: Int) + 2) |> ((x: Int) => x * 5 + x)
val res1: Int = 42 

Two Lambda Functions and 3 Pipes

This Python example employs 2 lambda functions and 3 pipes to add 10 to the even numbers from 0 to 5, exclusive.

Shell

>>> (
    range(5)
      | p(filter, _ % 2 == 0)
      | p(map, _ + 10)
      | p(list)
)
[10, 12, 14] 

Scala has a better way of performing this type of computation that does not require pipes or computation. It is better because it is simpler to understand.

Shell

scala> for {
     |   x <- (0 until 5).toList if x % 2 == 0
     |   y = x + 10
     | } yield y
val res12: List[Int] = List(10, 12, 14) 

Other examples of placeholder syntax

NumPy expressions (NumPy is Python-specific):

Shell

range(10) | np.sin(_)+1 | p(plt.plot)

Pandas expressions (Pandas is Python-specific):

Shell

people_df | _.loc[_.age > 10, 'name']

Solution for the 2nd Project Euler exercise:

Shell

>>> def fib():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

>>> euler2 = (
      fib()
        | p.where(_ % 2 == 0)
        | p.take_while(_ < 4000000)
        | p.add()
   )

>>> euler2
4613732

Looking Ahead to Scala 3 (Dotty)

The next major version of Scala, due out in a few months, will probably allow a Scala 3 extension method to define the vertical bar as a method for more readabile code:

Shell

def [A,B](a: A) |(f: (A) => B): B = a.pipe(f)

# Sample usage:
val x = 5 | doSomething | doSomethingElse | doSomethingMore

To Learn More

My Introduction to Scala course on ScalaCourses.com teaches Scala lambda functions.

Share to Twitter, Hacker News, LinkedIn.