assignment with lambda

Python Enhancement Proposals

• Python »
• PEP Index »

PEP 572 – Assignment Expressions

The importance of real code, exceptional cases, scope of the target, relative precedence of :=, change to evaluation order, differences between assignment expressions and assignment statements, specification changes during implementation, _pydecimal.py, datetime.py, sysconfig.py, simplifying list comprehensions, capturing condition values, changing the scope rules for comprehensions, alternative spellings, special-casing conditional statements, special-casing comprehensions, lowering operator precedence, allowing commas to the right, always requiring parentheses, why not just turn existing assignment into an expression, with assignment expressions, why bother with assignment statements, why not use a sublocal scope and prevent namespace pollution, style guide recommendations, acknowledgements, a numeric example, appendix b: rough code translations for comprehensions, appendix c: no changes to scope semantics.

This is a proposal for creating a way to assign to variables within an expression using the notation NAME := expr .

As part of this change, there is also an update to dictionary comprehension evaluation order to ensure key expressions are executed before value expressions (allowing the key to be bound to a name and then re-used as part of calculating the corresponding value).

During discussion of this PEP, the operator became informally known as “the walrus operator”. The construct’s formal name is “Assignment Expressions” (as per the PEP title), but they may also be referred to as “Named Expressions” (e.g. the CPython reference implementation uses that name internally).

Naming the result of an expression is an important part of programming, allowing a descriptive name to be used in place of a longer expression, and permitting reuse. Currently, this feature is available only in statement form, making it unavailable in list comprehensions and other expression contexts.

Additionally, naming sub-parts of a large expression can assist an interactive debugger, providing useful display hooks and partial results. Without a way to capture sub-expressions inline, this would require refactoring of the original code; with assignment expressions, this merely requires the insertion of a few name := markers. Removing the need to refactor reduces the likelihood that the code be inadvertently changed as part of debugging (a common cause of Heisenbugs), and is easier to dictate to another programmer.

During the development of this PEP many people (supporters and critics both) have had a tendency to focus on toy examples on the one hand, and on overly complex examples on the other.

The danger of toy examples is twofold: they are often too abstract to make anyone go “ooh, that’s compelling”, and they are easily refuted with “I would never write it that way anyway”.

The danger of overly complex examples is that they provide a convenient strawman for critics of the proposal to shoot down (“that’s obfuscated”).

Yet there is some use for both extremely simple and extremely complex examples: they are helpful to clarify the intended semantics. Therefore, there will be some of each below.

However, in order to be compelling , examples should be rooted in real code, i.e. code that was written without any thought of this PEP, as part of a useful application, however large or small. Tim Peters has been extremely helpful by going over his own personal code repository and picking examples of code he had written that (in his view) would have been clearer if rewritten with (sparing) use of assignment expressions. His conclusion: the current proposal would have allowed a modest but clear improvement in quite a few bits of code.

Another use of real code is to observe indirectly how much value programmers place on compactness. Guido van Rossum searched through a Dropbox code base and discovered some evidence that programmers value writing fewer lines over shorter lines.

Case in point: Guido found several examples where a programmer repeated a subexpression, slowing down the program, in order to save one line of code, e.g. instead of writing:

they would write:

Another example illustrates that programmers sometimes do more work to save an extra level of indentation:

This code tries to match pattern2 even if pattern1 has a match (in which case the match on pattern2 is never used). The more efficient rewrite would have been:

Syntax and semantics

In most contexts where arbitrary Python expressions can be used, a named expression can appear. This is of the form NAME := expr where expr is any valid Python expression other than an unparenthesized tuple, and NAME is an identifier.

The value of such a named expression is the same as the incorporated expression, with the additional side-effect that the target is assigned that value:

There are a few places where assignment expressions are not allowed, in order to avoid ambiguities or user confusion:

This rule is included to simplify the choice for the user between an assignment statement and an assignment expression – there is no syntactic position where both are valid.

Again, this rule is included to avoid two visually similar ways of saying the same thing.

This rule is included to disallow excessively confusing code, and because parsing keyword arguments is complex enough already.

This rule is included to discourage side effects in a position whose exact semantics are already confusing to many users (cf. the common style recommendation against mutable default values), and also to echo the similar prohibition in calls (the previous bullet).

The reasoning here is similar to the two previous cases; this ungrouped assortment of symbols and operators composed of : and = is hard to read correctly.

This allows lambda to always bind less tightly than := ; having a name binding at the top level inside a lambda function is unlikely to be of value, as there is no way to make use of it. In cases where the name will be used more than once, the expression is likely to need parenthesizing anyway, so this prohibition will rarely affect code.

This shows that what looks like an assignment operator in an f-string is not always an assignment operator. The f-string parser uses : to indicate formatting options. To preserve backwards compatibility, assignment operator usage inside of f-strings must be parenthesized. As noted above, this usage of the assignment operator is not recommended.

An assignment expression does not introduce a new scope. In most cases the scope in which the target will be bound is self-explanatory: it is the current scope. If this scope contains a nonlocal or global declaration for the target, the assignment expression honors that. A lambda (being an explicit, if anonymous, function definition) counts as a scope for this purpose.

There is one special case: an assignment expression occurring in a list, set or dict comprehension or in a generator expression (below collectively referred to as “comprehensions”) binds the target in the containing scope, honoring a nonlocal or global declaration for the target in that scope, if one exists. For the purpose of this rule the containing scope of a nested comprehension is the scope that contains the outermost comprehension. A lambda counts as a containing scope.

The motivation for this special case is twofold. First, it allows us to conveniently capture a “witness” for an any() expression, or a counterexample for all() , for example:

Second, it allows a compact way of updating mutable state from a comprehension, for example:

However, an assignment expression target name cannot be the same as a for -target name appearing in any comprehension containing the assignment expression. The latter names are local to the comprehension in which they appear, so it would be contradictory for a contained use of the same name to refer to the scope containing the outermost comprehension instead.

For example, [i := i+1 for i in range(5)] is invalid: the for i part establishes that i is local to the comprehension, but the i := part insists that i is not local to the comprehension. The same reason makes these examples invalid too:

While it’s technically possible to assign consistent semantics to these cases, it’s difficult to determine whether those semantics actually make sense in the absence of real use cases. Accordingly, the reference implementation [1] will ensure that such cases raise SyntaxError , rather than executing with implementation defined behaviour.

This restriction applies even if the assignment expression is never executed:

For the comprehension body (the part before the first “for” keyword) and the filter expression (the part after “if” and before any nested “for”), this restriction applies solely to target names that are also used as iteration variables in the comprehension. Lambda expressions appearing in these positions introduce a new explicit function scope, and hence may use assignment expressions with no additional restrictions.

Due to design constraints in the reference implementation (the symbol table analyser cannot easily detect when names are re-used between the leftmost comprehension iterable expression and the rest of the comprehension), named expressions are disallowed entirely as part of comprehension iterable expressions (the part after each “in”, and before any subsequent “if” or “for” keyword):

A further exception applies when an assignment expression occurs in a comprehension whose containing scope is a class scope. If the rules above were to result in the target being assigned in that class’s scope, the assignment expression is expressly invalid. This case also raises SyntaxError :

(The reason for the latter exception is the implicit function scope created for comprehensions – there is currently no runtime mechanism for a function to refer to a variable in the containing class scope, and we do not want to add such a mechanism. If this issue ever gets resolved this special case may be removed from the specification of assignment expressions. Note that the problem already exists for using a variable defined in the class scope from a comprehension.)

See Appendix B for some examples of how the rules for targets in comprehensions translate to equivalent code.

The := operator groups more tightly than a comma in all syntactic positions where it is legal, but less tightly than all other operators, including or , and , not , and conditional expressions ( A if C else B ). As follows from section “Exceptional cases” above, it is never allowed at the same level as = . In case a different grouping is desired, parentheses should be used.

The := operator may be used directly in a positional function call argument; however it is invalid directly in a keyword argument.

Some examples to clarify what’s technically valid or invalid:

Most of the “valid” examples above are not recommended, since human readers of Python source code who are quickly glancing at some code may miss the distinction. But simple cases are not objectionable:

This PEP recommends always putting spaces around := , similar to PEP 8 ’s recommendation for = when used for assignment, whereas the latter disallows spaces around = used for keyword arguments.)

In order to have precisely defined semantics, the proposal requires evaluation order to be well-defined. This is technically not a new requirement, as function calls may already have side effects. Python already has a rule that subexpressions are generally evaluated from left to right. However, assignment expressions make these side effects more visible, and we propose a single change to the current evaluation order:

• In a dict comprehension {X: Y for ...} , Y is currently evaluated before X . We propose to change this so that X is evaluated before Y . (In a dict display like {X: Y} this is already the case, and also in dict((X, Y) for ...) which should clearly be equivalent to the dict comprehension.)

Most importantly, since := is an expression, it can be used in contexts where statements are illegal, including lambda functions and comprehensions.

Conversely, assignment expressions don’t support the advanced features found in assignment statements:

• Multiple targets are not directly supported: x = y = z = 0 # Equivalent: (z := (y := (x := 0)))
• Single assignment targets other than a single NAME are not supported: # No equivalent a [ i ] = x self . rest = []
• Priority around commas is different: x = 1 , 2 # Sets x to (1, 2) ( x := 1 , 2 ) # Sets x to 1
• Iterable packing and unpacking (both regular or extended forms) are not supported: # Equivalent needs extra parentheses loc = x , y # Use (loc := (x, y)) info = name , phone , * rest # Use (info := (name, phone, *rest)) # No equivalent px , py , pz = position name , phone , email , * other_info = contact
• Inline type annotations are not supported: # Closest equivalent is "p: Optional[int]" as a separate declaration p : Optional [ int ] = None
• Augmented assignment is not supported: total += tax # Equivalent: (total := total + tax)

The following changes have been made based on implementation experience and additional review after the PEP was first accepted and before Python 3.8 was released:

• for consistency with other similar exceptions, and to avoid locking in an exception name that is not necessarily going to improve clarity for end users, the originally proposed TargetScopeError subclass of SyntaxError was dropped in favour of just raising SyntaxError directly. [3]
• due to a limitation in CPython’s symbol table analysis process, the reference implementation raises SyntaxError for all uses of named expressions inside comprehension iterable expressions, rather than only raising them when the named expression target conflicts with one of the iteration variables in the comprehension. This could be revisited given sufficiently compelling examples, but the extra complexity needed to implement the more selective restriction doesn’t seem worthwhile for purely hypothetical use cases.

Examples from the Python standard library

env_base is only used on these lines, putting its assignment on the if moves it as the “header” of the block.

• Current: env_base = os . environ . get ( "PYTHONUSERBASE" , None ) if env_base : return env_base
• Improved: if env_base := os . environ . get ( "PYTHONUSERBASE" , None ): return env_base

Avoid nested if and remove one indentation level.

• Current: if self . _is_special : ans = self . _check_nans ( context = context ) if ans : return ans
• Improved: if self . _is_special and ( ans := self . _check_nans ( context = context )): return ans

Code looks more regular and avoid multiple nested if. (See Appendix A for the origin of this example.)

• Current: reductor = dispatch_table . get ( cls ) if reductor : rv = reductor ( x ) else : reductor = getattr ( x , "__reduce_ex__" , None ) if reductor : rv = reductor ( 4 ) else : reductor = getattr ( x , "__reduce__" , None ) if reductor : rv = reductor () else : raise Error ( "un(deep)copyable object of type %s " % cls )
• Improved: if reductor := dispatch_table . get ( cls ): rv = reductor ( x ) elif reductor := getattr ( x , "__reduce_ex__" , None ): rv = reductor ( 4 ) elif reductor := getattr ( x , "__reduce__" , None ): rv = reductor () else : raise Error ( "un(deep)copyable object of type %s " % cls )

tz is only used for s += tz , moving its assignment inside the if helps to show its scope.

• Current: s = _format_time ( self . _hour , self . _minute , self . _second , self . _microsecond , timespec ) tz = self . _tzstr () if tz : s += tz return s
• Improved: s = _format_time ( self . _hour , self . _minute , self . _second , self . _microsecond , timespec ) if tz := self . _tzstr (): s += tz return s

Calling fp.readline() in the while condition and calling .match() on the if lines make the code more compact without making it harder to understand.

• Current: while True : line = fp . readline () if not line : break m = define_rx . match ( line ) if m : n , v = m . group ( 1 , 2 ) try : v = int ( v ) except ValueError : pass vars [ n ] = v else : m = undef_rx . match ( line ) if m : vars [ m . group ( 1 )] = 0
• Improved: while line := fp . readline (): if m := define_rx . match ( line ): n , v = m . group ( 1 , 2 ) try : v = int ( v ) except ValueError : pass vars [ n ] = v elif m := undef_rx . match ( line ): vars [ m . group ( 1 )] = 0

A list comprehension can map and filter efficiently by capturing the condition:

Similarly, a subexpression can be reused within the main expression, by giving it a name on first use:

Note that in both cases the variable y is bound in the containing scope (i.e. at the same level as results or stuff ).

Assignment expressions can be used to good effect in the header of an if or while statement:

Particularly with the while loop, this can remove the need to have an infinite loop, an assignment, and a condition. It also creates a smooth parallel between a loop which simply uses a function call as its condition, and one which uses that as its condition but also uses the actual value.

An example from the low-level UNIX world:

Rejected alternative proposals

Proposals broadly similar to this one have come up frequently on python-ideas. Below are a number of alternative syntaxes, some of them specific to comprehensions, which have been rejected in favour of the one given above.

A previous version of this PEP proposed subtle changes to the scope rules for comprehensions, to make them more usable in class scope and to unify the scope of the “outermost iterable” and the rest of the comprehension. However, this part of the proposal would have caused backwards incompatibilities, and has been withdrawn so the PEP can focus on assignment expressions.

Broadly the same semantics as the current proposal, but spelled differently.

Since EXPR as NAME already has meaning in import , except and with statements (with different semantics), this would create unnecessary confusion or require special-casing (e.g. to forbid assignment within the headers of these statements).

(Note that with EXPR as VAR does not simply assign the value of EXPR to VAR – it calls EXPR.__enter__() and assigns the result of that to VAR .)

Additional reasons to prefer := over this spelling include:

• In if f(x) as y the assignment target doesn’t jump out at you – it just reads like if f x blah blah and it is too similar visually to if f(x) and y .
• import foo as bar
• except Exc as var
• with ctxmgr() as var

To the contrary, the assignment expression does not belong to the if or while that starts the line, and we intentionally allow assignment expressions in other contexts as well.

• NAME = EXPR
• if NAME := EXPR

reinforces the visual recognition of assignment expressions.

This syntax is inspired by languages such as R and Haskell, and some programmable calculators. (Note that a left-facing arrow y <- f(x) is not possible in Python, as it would be interpreted as less-than and unary minus.) This syntax has a slight advantage over ‘as’ in that it does not conflict with with , except and import , but otherwise is equivalent. But it is entirely unrelated to Python’s other use of -> (function return type annotations), and compared to := (which dates back to Algol-58) it has a much weaker tradition.

This has the advantage that leaked usage can be readily detected, removing some forms of syntactic ambiguity. However, this would be the only place in Python where a variable’s scope is encoded into its name, making refactoring harder.

Execution order is inverted (the indented body is performed first, followed by the “header”). This requires a new keyword, unless an existing keyword is repurposed (most likely with: ). See PEP 3150 for prior discussion on this subject (with the proposed keyword being given: ).

This syntax has fewer conflicts than as does (conflicting only with the raise Exc from Exc notation), but is otherwise comparable to it. Instead of paralleling with expr as target: (which can be useful but can also be confusing), this has no parallels, but is evocative.

One of the most popular use-cases is if and while statements. Instead of a more general solution, this proposal enhances the syntax of these two statements to add a means of capturing the compared value:

This works beautifully if and ONLY if the desired condition is based on the truthiness of the captured value. It is thus effective for specific use-cases (regex matches, socket reads that return '' when done), and completely useless in more complicated cases (e.g. where the condition is f(x) < 0 and you want to capture the value of f(x) ). It also has no benefit to list comprehensions.

Advantages: No syntactic ambiguities. Disadvantages: Answers only a fraction of possible use-cases, even in if / while statements.

Another common use-case is comprehensions (list/set/dict, and genexps). As above, proposals have been made for comprehension-specific solutions.

This brings the subexpression to a location in between the ‘for’ loop and the expression. It introduces an additional language keyword, which creates conflicts. Of the three, where reads the most cleanly, but also has the greatest potential for conflict (e.g. SQLAlchemy and numpy have where methods, as does tkinter.dnd.Icon in the standard library).

As above, but reusing the with keyword. Doesn’t read too badly, and needs no additional language keyword. Is restricted to comprehensions, though, and cannot as easily be transformed into “longhand” for-loop syntax. Has the C problem that an equals sign in an expression can now create a name binding, rather than performing a comparison. Would raise the question of why “with NAME = EXPR:” cannot be used as a statement on its own.

As per option 2, but using as rather than an equals sign. Aligns syntactically with other uses of as for name binding, but a simple transformation to for-loop longhand would create drastically different semantics; the meaning of with inside a comprehension would be completely different from the meaning as a stand-alone statement, while retaining identical syntax.

Regardless of the spelling chosen, this introduces a stark difference between comprehensions and the equivalent unrolled long-hand form of the loop. It is no longer possible to unwrap the loop into statement form without reworking any name bindings. The only keyword that can be repurposed to this task is with , thus giving it sneakily different semantics in a comprehension than in a statement; alternatively, a new keyword is needed, with all the costs therein.

There are two logical precedences for the := operator. Either it should bind as loosely as possible, as does statement-assignment; or it should bind more tightly than comparison operators. Placing its precedence between the comparison and arithmetic operators (to be precise: just lower than bitwise OR) allows most uses inside while and if conditions to be spelled without parentheses, as it is most likely that you wish to capture the value of something, then perform a comparison on it:

Once find() returns -1, the loop terminates. If := binds as loosely as = does, this would capture the result of the comparison (generally either True or False ), which is less useful.

While this behaviour would be convenient in many situations, it is also harder to explain than “the := operator behaves just like the assignment statement”, and as such, the precedence for := has been made as close as possible to that of = (with the exception that it binds tighter than comma).

Some critics have claimed that the assignment expressions should allow unparenthesized tuples on the right, so that these two would be equivalent:

(With the current version of the proposal, the latter would be equivalent to ((point := x), y) .)

However, adopting this stance would logically lead to the conclusion that when used in a function call, assignment expressions also bind less tight than comma, so we’d have the following confusing equivalence:

The less confusing option is to make := bind more tightly than comma.

It’s been proposed to just always require parentheses around an assignment expression. This would resolve many ambiguities, and indeed parentheses will frequently be needed to extract the desired subexpression. But in the following cases the extra parentheses feel redundant:

Frequently Raised Objections

C and its derivatives define the = operator as an expression, rather than a statement as is Python’s way. This allows assignments in more contexts, including contexts where comparisons are more common. The syntactic similarity between if (x == y) and if (x = y) belies their drastically different semantics. Thus this proposal uses := to clarify the distinction.

The two forms have different flexibilities. The := operator can be used inside a larger expression; the = statement can be augmented to += and its friends, can be chained, and can assign to attributes and subscripts.

Previous revisions of this proposal involved sublocal scope (restricted to a single statement), preventing name leakage and namespace pollution. While a definite advantage in a number of situations, this increases complexity in many others, and the costs are not justified by the benefits. In the interests of language simplicity, the name bindings created here are exactly equivalent to any other name bindings, including that usage at class or module scope will create externally-visible names. This is no different from for loops or other constructs, and can be solved the same way: del the name once it is no longer needed, or prefix it with an underscore.

(The author wishes to thank Guido van Rossum and Christoph Groth for their suggestions to move the proposal in this direction. [2] )

As expression assignments can sometimes be used equivalently to statement assignments, the question of which should be preferred will arise. For the benefit of style guides such as PEP 8 , two recommendations are suggested.

• If either assignment statements or assignment expressions can be used, prefer statements; they are a clear declaration of intent.
• If using assignment expressions would lead to ambiguity about execution order, restructure it to use statements instead.

The authors wish to thank Alyssa Coghlan and Steven D’Aprano for their considerable contributions to this proposal, and members of the core-mentorship mailing list for assistance with implementation.

Appendix A: Tim Peters’s findings

Here’s a brief essay Tim Peters wrote on the topic.

I dislike “busy” lines of code, and also dislike putting conceptually unrelated logic on a single line. So, for example, instead of:

instead. So I suspected I’d find few places I’d want to use assignment expressions. I didn’t even consider them for lines already stretching halfway across the screen. In other cases, “unrelated” ruled:

is a vast improvement over the briefer:

The original two statements are doing entirely different conceptual things, and slamming them together is conceptually insane.

In other cases, combining related logic made it harder to understand, such as rewriting:

as the briefer:

The while test there is too subtle, crucially relying on strict left-to-right evaluation in a non-short-circuiting or method-chaining context. My brain isn’t wired that way.

But cases like that were rare. Name binding is very frequent, and “sparse is better than dense” does not mean “almost empty is better than sparse”. For example, I have many functions that return None or 0 to communicate “I have nothing useful to return in this case, but since that’s expected often I’m not going to annoy you with an exception”. This is essentially the same as regular expression search functions returning None when there is no match. So there was lots of code of the form:

I find that clearer, and certainly a bit less typing and pattern-matching reading, as:

It’s also nice to trade away a small amount of horizontal whitespace to get another _line_ of surrounding code on screen. I didn’t give much weight to this at first, but it was so very frequent it added up, and I soon enough became annoyed that I couldn’t actually run the briefer code. That surprised me!

There are other cases where assignment expressions really shine. Rather than pick another from my code, Kirill Balunov gave a lovely example from the standard library’s copy() function in copy.py :

The ever-increasing indentation is semantically misleading: the logic is conceptually flat, “the first test that succeeds wins”:

Using easy assignment expressions allows the visual structure of the code to emphasize the conceptual flatness of the logic; ever-increasing indentation obscured it.

A smaller example from my code delighted me, both allowing to put inherently related logic in a single line, and allowing to remove an annoying “artificial” indentation level:

That if is about as long as I want my lines to get, but remains easy to follow.

So, in all, in most lines binding a name, I wouldn’t use assignment expressions, but because that construct is so very frequent, that leaves many places I would. In most of the latter, I found a small win that adds up due to how often it occurs, and in the rest I found a moderate to major win. I’d certainly use it more often than ternary if , but significantly less often than augmented assignment.

I have another example that quite impressed me at the time.

Where all variables are positive integers, and a is at least as large as the n’th root of x, this algorithm returns the floor of the n’th root of x (and roughly doubling the number of accurate bits per iteration):

It’s not obvious why that works, but is no more obvious in the “loop and a half” form. It’s hard to prove correctness without building on the right insight (the “arithmetic mean - geometric mean inequality”), and knowing some non-trivial things about how nested floor functions behave. That is, the challenges are in the math, not really in the coding.

If you do know all that, then the assignment-expression form is easily read as “while the current guess is too large, get a smaller guess”, where the “too large?” test and the new guess share an expensive sub-expression.

To my eyes, the original form is harder to understand:

This appendix attempts to clarify (though not specify) the rules when a target occurs in a comprehension or in a generator expression. For a number of illustrative examples we show the original code, containing a comprehension, and the translation, where the comprehension has been replaced by an equivalent generator function plus some scaffolding.

Since [x for ...] is equivalent to list(x for ...) these examples all use list comprehensions without loss of generality. And since these examples are meant to clarify edge cases of the rules, they aren’t trying to look like real code.

Note: comprehensions are already implemented via synthesizing nested generator functions like those in this appendix. The new part is adding appropriate declarations to establish the intended scope of assignment expression targets (the same scope they resolve to as if the assignment were performed in the block containing the outermost comprehension). For type inference purposes, these illustrative expansions do not imply that assignment expression targets are always Optional (but they do indicate the target binding scope).

Let’s start with a reminder of what code is generated for a generator expression without assignment expression.

• Original code (EXPR usually references VAR): def f (): a = [ EXPR for VAR in ITERABLE ]
• Translation (let’s not worry about name conflicts): def f (): def genexpr ( iterator ): for VAR in iterator : yield EXPR a = list ( genexpr ( iter ( ITERABLE )))

Let’s add a simple assignment expression.

• Original code: def f (): a = [ TARGET := EXPR for VAR in ITERABLE ]
• Translation: def f (): if False : TARGET = None # Dead code to ensure TARGET is a local variable def genexpr ( iterator ): nonlocal TARGET for VAR in iterator : TARGET = EXPR yield TARGET a = list ( genexpr ( iter ( ITERABLE )))

Let’s add a global TARGET declaration in f() .

• Original code: def f (): global TARGET a = [ TARGET := EXPR for VAR in ITERABLE ]
• Translation: def f (): global TARGET def genexpr ( iterator ): global TARGET for VAR in iterator : TARGET = EXPR yield TARGET a = list ( genexpr ( iter ( ITERABLE )))

Or instead let’s add a nonlocal TARGET declaration in f() .

• Original code: def g (): TARGET = ... def f (): nonlocal TARGET a = [ TARGET := EXPR for VAR in ITERABLE ]
• Translation: def g (): TARGET = ... def f (): nonlocal TARGET def genexpr ( iterator ): nonlocal TARGET for VAR in iterator : TARGET = EXPR yield TARGET a = list ( genexpr ( iter ( ITERABLE )))

Finally, let’s nest two comprehensions.

• Original code: def f (): a = [[ TARGET := i for i in range ( 3 )] for j in range ( 2 )] # I.e., a = [[0, 1, 2], [0, 1, 2]] print ( TARGET ) # prints 2
• Translation: def f (): if False : TARGET = None def outer_genexpr ( outer_iterator ): nonlocal TARGET def inner_generator ( inner_iterator ): nonlocal TARGET for i in inner_iterator : TARGET = i yield i for j in outer_iterator : yield list ( inner_generator ( range ( 3 ))) a = list ( outer_genexpr ( range ( 2 ))) print ( TARGET )

Because it has been a point of confusion, note that nothing about Python’s scoping semantics is changed. Function-local scopes continue to be resolved at compile time, and to have indefinite temporal extent at run time (“full closures”). Example:

This document has been placed in the public domain.

Source: https://github.com/python/peps/blob/main/peps/pep-0572.rst

Last modified: 2023-10-11 12:05:51 GMT

Lambda Expressions

A guide to programming lambda expressions in C++, C#, Java, Javascript, and Python by JoshData .

A lambda expression is a convenient syntax available in many programming languages for writing short functions. Lambda expressions do for functions what object-oriented programing does for objects: It makes a function something you can assign to a variable.

Lambda expressions are also sometimes called anonymous functions, lambda functions, and closures. The capabilities of lambda expressions differs across programming languages, and some programming languages have multiple ways of writing lambda expressions. The concept originates in philosophy.

Background Concepts

Let’s cover some general programming concepts first.

Statements versus expressions

All programming languages with lambda expressions distinguish statements from expressions :

• A statement is typically a line of code. Statements control the flow of the program , like if statements, for loops, calling subroutines, and return statements in functions. An assignment to a variable (e.g. x = y ) is also a statement.
• An expression is code within a statement that computes a value . Expressions are made up of constants, variables, arithmetic operators, function calls, method calls, property accesors, and so on. An expression encodes a set of instructions that results in the computation of one value.

if x > y is a statement. x > y is an expression — it computes one value, in this case either true or false. The if statement takes an action based on the value of the expression.

Statically versus dynamically typed languages

This guide covers a few common programming languages, and if you are reading it to learn about multiple languages it may be helpful to distinguish:

• Statically typed languages, meaning all variables are defined with a type in source code that is known to the compiler. C, C++, C#, and Java are statically typed languages using types such as int , double , string , etc.
• Dynamically typed languages, meaning any variable can hold any type of value. Variable declarations with types do not occur in these languages. Python and Javascript are dynamically typed. These languages still have types, usually the same types as in the C-family languages, but they’re only for the values computed in memory, not for variables in source code.

Lambda Syntax

A lambda expression is a function written in a shorthand syntax, and we’ll start by looking at the syntax of lambda expressions.

Here is a regular function (not a lambda function, just a normal function) that adds one to a number as it would be written in C, C++, C#, and Java (they all happen to be identical in this case) and in Javascript. We’ll look at how the same function would be written as a lambda expression.

In this example, there is a single argument x , the expression x + 1 is computed, and that expression is returned to the caller.

It can be written shorter as a lambda expression in modern versions of many languages, including C++ (starting in C++11), C# (starting in C# 9.0), Java (since Java 8), Javascript (starting in ECMAScript 6), and Python (since around Python 2.2):

C#, Java, and Javascript use an arrow symbol (made up of an equal sign or dash plus a greater than sign) to separate the arguments from the expression body. The C# and Javascript syntaxes happen to be identical in this case. (In Javascript, an “anonymous function” refers to a different but related syntactic structure — the technical name for the Javascript syntax discussed here is the “arrow function expression.”)

In the academic discipline of philosophy, the function would be written with the Greek lowercase letter lambda λ denoting the start of a lambda expression:

Here’s a function that returns the maximum value of two numbers, first written as a regular function:

and then as a lambda expression in various languages:

Lambda expressions can also have zero arguments:

Lambda expressions have most of the same components as a normal function:

• An argument list: Zero or more variables separated with commas, usually in parentheses. C#, Java, and Javascript allow omitting parentheses if there is only one argument. C++ requires the arguments to have types, but the other statically typed languages allow it without making it required.
• A body: Statements to execute or an expression to evaluate and return. When using statements, they are surrounded with braces, and a return statement can be used to return a value. When the body is just an expression, braces are omitted. (C++ only allows using statements and Python only allows using expressions. In C#, braces can be omitted when the body is a single statement.)

In the body of the lambda expression, some languages allow both statement blocks and expressions (C#), some only support statements (C++), and some only support expressions (Python). The add_one examples above used expressions as lambda function bodies where possible: x + 1 in C#, Python, etc. No braces are used in those languages when the body is an expression:

The max examples above used statement blocks as lambda function bodies, and when statements are used these languages require that they be surrounded in braces:

This last example could be turned into an expression-bodied lambda by using the ternary operator condition ? then : else . Expression bodies are usually easier to read because they are more compact.

The one thing missing from lambda expressions is a function name. Lambda expressions don’t have a place for a name in their syntax.

(No type is given for the return value in any of the examples above either. It is optional in C++ and not permitted in other languages. In statically typed languages, the lambda expression does have a return type but the compiler figures it out from what you return.)

Lambda Expressions are Expressions

The key difference between a regularly defined function and a lambda expression is where they occur in source code. Regular functions are normally defined either at the top-level of your source code file or in a class. All of the statements in your source code are contained within a function.

A lambda expression is exactly the opposite. It occurs not outside of statements but inside statements. It actually is an expression , which means it occurs within statements wherever an expression can be used.

Assigning a lambda expression to a variable

It can be assigned to a variable. Variable assignment normally looks like this of course:

(C++ and C# are a statically typed language so the variable must be declared first with its type, but we will omit the variable declaration for now.)

Now instead of 10 , the variable will be assigned a lambda expression:

The lambda expression is assigned to the variable f . Note that the f = at the start and the semicolon ; at the end belong to the variable assignment statement . The lambda expression appears in the middle as the expression. Since the lambda expression is multiple lines, it pushes the semicolon ; down a few lines to its end.

After the variable assignment, the variable actually holds the lambda function defined by the lambda expression.

Quick look at the type of lambda expressions in the statically typed languages

You can also make a variable declaration with an explicit type, and in C# ≤9 and Java it’s required, even though both have var keywords. The variable types to use in C++, C# and Java are:

• C++: std::function<double(double, double)> (with #include <functional> )
• C#: System.Func<double, double, double>
• Java: java.util.function.BinaryOperator<Double>

For example, in Java:

We’ll come back to the type of lambda expressions later.

No type is needed in dynamically typed languages like Javascript and Python.

Few meaningful operators are defined for lambda expressions

Lambda expressions are expressions, but most operators don’t have any meaning with the value that a lambda expression computes. You can’t add two lambda functions together with + : It doesn’t make sense and that isn’t defined in any language.

The first operator that can be used with lambda expressions are grouping parentheses. Lambda expressions can always be wrapped in parentheses for clarity:

Be sure you see where the open and closing parens were added here.

Calling Lambda Expressions

The most important operator that works with lambda functions is the call operator. The next example starts with the same variable assignment as above, but it follows with a new statement that invokes the lambda function by calling it:

It is just like calling a function, but instead of f being the name of a function it is the name of a variable holding a lambda function.

When f(10, 20) is called, control flow moves to the lambda expression. The statements of the lambda expression are evaluated until the return statement is executed. Then control flow moves back the assignment of the value to z .

Assigning a lambda expression to a variable and then, some time later, using the call operator on the variable is the main thing lambda expressions do.

Java does not support the call operator — this is unusual for a language that has lambda expressions. To invoke the lambda function, in this example we use .apply(...) which is the correct method to use with BinaryOperator<Double>. (The method name depends on the runnable interface type that the expression is assigned to.)

In the next toy example, f is first set to a lambda expression that computes the maximum value of two arguments and then later it is set to a different lambda expression that computes the minimum value of two arguments. Although the same code f(10, 20) is called identically after each assignment, f returns a different value each time because it executes the two lambda functions:

Although f(10, 20) appears twice, it computes a different value each time. In the first call to f , control flow goes to the first lambda expression. In the second call to f , control flow goes to the second lambda expression.

In some languages, the call operator (10, 20) can occur after any expresson that evaluates to a lambda function. In C++, Javascript, and Python, it can be right after the lambda expression itself:

Make sure you see where the lambda expression begins and ends and where the call operator begins and ends.

This pattern is commonly seen in Javascript for reasons related to scope and not really about the lambda expression. It isn’t generally useful because you can always write an expression like this without the complex lambda expression syntax.

C# and Java do not allow this.

Passing lambda expressions as arguments to other functions

The last meaningful thing that you can do with lambda expressions is passing the lambda expresson as an argument to a function. In the next set of examples, a new function foo is defined that takes one argument. The program calls foo and passes a lambda expression as the argment.

Here are additional examples using the more complex lambda functions with statement bodies:

When passing lambda expressions with statement bodies in function calls, the triple of symbols brace-paren-semicolon } ); is a common line ending. The close brace belongs to the lambda expression: It is the close-brace that ends the lambda expression’s body. The close parenthesis belongs to the function call: It is the close parethesis at the end of the argument list. The semicolon marks the end of the statement as in all C++, C#, Java, and Javascript statements.

The most common way to use a lambda expression is passing it to another function and then calling it within that function. Here is the same function call example again, plus calling the lambda function within foo :

Finally, we look at a lambda expression with no return value. Lambda expressions without a return value are typically used with statement-block bodies to take an action.

This lambda expression has a void return type in C++, C#, and Java. (Javascript and Python do not support void return types — if there is no return value, the lambda expression returns undefined (Javascript) or None (Python).)

In C++, C#, Java, Javascript, and Python, any regular function name or class method can also be assigned to a variable and passed to a function, like lambda expressions. In the statically typed languages, the variable or function argument must have the right type. But in dynamically typed languages, that’s not an issue and passing around functions can be very natural:

Terminology

In this guide, lambda expression and lambda function mean slightly different things, although I can’t promise that anyone else makes this distinction:

A lambda expression is the code you type to define a short function. It is source code text that goes into the compiler and is recognized with a particular syntax. (In Javascript, technically they are called arrow function expressions/declarations.)

The expression evaluates at run time to a lambda function in memory. In memory during program execution, the f variable in the preceding examples holds a lambda function . It doesn’t hold the source code text that you typed in — that’s been compiled into some other more efficient representation. So it doesn’t hold an expression. Instead it probably holds a pointer to a memory location that has the compiled code.

The difference between a lambda expression and a lambda function is similar to the difference between a class and an instance of the class (an object). A class is a definition of a type of object. At run time, variables whose types are classes don’t hold classes: they hold pointers to objects. Similarly, variables that are assigned lambda expressions in code hold pointers to lambda functions at run time, not lambda expressions. (In fact, in many languages the lambda expression actually compiles to a new instance of a hidden class!)

Standard Library Routines that Use Lambda Functions

The standard library in each programming language has some methods that are convenient to use with lambda expressions.

Across programming languages, lambda functions are commonly used with the language’s standard library sort function to create sort orders for user-defined data types. In C++, a lambda expression can be passed to std::sort for this purpose.

std::sort ’s third argument is a function that compares two items in the list and returns whether the first item should come first. The arguments to the comparison function must be const references.

In this example, a user-defined class is sorted first by its name field and then, when there are any instances with the same name, by its value field.

std::sort will call the lambda function for each pair of elements in the list and will use its return value to sort the elements according to the order that the lambda function defines. The comparison function always looks something like this to achieve a sort order over multiple fields.

The standard library has a handful of functions that take comparison functions like sort does, including min_element / max_element — another common use of lambda functions across languages. This example finds the MyClass instance in a vector with the smallest value .

This is more compact than writing a for loop to iterate over the elements and track which one has the minimum value.

A comparison function can be used to create set and map containers for user-defined data types. (A hash code generator can also be used with unordered_set and unordered_map .)

You can also put lambda functions inside containers:

Across programming languages, lambda functions are commonly used with the language’s container sort functions to create sort orders for user-defined data types. In C#, a lambda expression can be passed to List.Sort(...) for this purpose.

List.Sort() ’s optional argument is a function that compares two items in the list and returns which should be first in the list. The comparison function returns -1 if the first item should come first, 1 if the second item should come first, or 0 if the items can have either order.

List.Sort will call the lambda function for each pair of elements in the list and will use its return value to sort the elements according to the order that the lambda function defines. The comparison function always looks something like this to achieve a sort order over multiple fields. String.CompareTo and Double.CompareTo have the same sematics as the function expected by Sort: They return -1, 0, or 1.

List.ForEach is another helpful method with a lambda expression — it simply runs the function on each element of the list. Here’s how you can print out each item in the list:

You could of course also write a regular foreach loop, but the lambda expression syntax might be clearer or cleaner in some cases.

You can also put lambda functions inside lists:

The extension methods in System.Linq.Enumerable ( reference ) provide other utility methods on collections that are helpful when used with lambda expressions. For example, Count can be used to count the items in a list that pass a test:

Lambda expressions are also commonly used with C# events, such as in System.Windows.Forms applications.

Rather than subscribing methods to event handlers (which are often hooked up by the Visual Studio designer automatically):

A lambda expression could be used instead:

Don’t subscribe to events directly with lambda expressions if the event handler (the lambda function) needs to be unsubscribed from the event later. To do that, you would need to assign the lambda expression to a variable first and then later use the variable to unsubscribe from the event.

System.Threading.Tasks.Task can be used to launch a background task that runs asynchronously on a thread pool and System.Threading.Tasks.Parallel can launch many tasks at once on the thread pool. Lambda expressions are convenient for both.

First, a single background task:

Next, a background task is launched for each item in an array:

The lambda expression is run multiple times, possibly simultaneously, for each item in the array, and the order in which the array elements are seen might be unpredictable. The first argument to ForEach can be any IEnumerable container. Unlike Task.Run which returns immediately, Parallel.ForEach waits until all of the loop iterations complete.

See https://ddc-java-10.github.io/2020/04/28/lambdas-key-functional-interfaces/ for some examples.

Lambda expressions and anonymous functions are used extensively in Javascript, in two ways:

• When a lambda expression (let’s call it L) is passed as an argument to a function (let’s call it F) and the lambda L is executed by function F immediately, before the function F returns, the call to the lambda L is synchronous . All of the Javascript examples in this guide so far have been synchronous.
• When the function F stores the lambda L to be called later, so that F can return before L is executed, the call to the lambda L is asynchronous . It often happens after other stored lambda functions are executed, creating a sort of concurrent execution of multiple tasks. An asynchronously executed lambda expression is often called a “callback.”

Asynchronous callbacks are so pervasive in Javascript that I can’t even begin here to provide key examples. The async package and the Promise design concept are the key places to look next.

Across programming languages, lambda functions are commonly used with the language’s list sorting function to create sort orders for user-defined classes for which the language doesn’t provide any built-in ordering. In Python, a lambda expression can be passed to list.sort() and sorted(...) for this purpose.

You might have seen this error when trying to sort user-defined classes:

MyClass , in this example, is not a sortable data type. You could implement the < operator on MyClass to make it sortable, but often the easier solution is to call list.sort with a lambda expression.

list.sort() ’s optional keyword argument key takes a function that takes an item of the list and returns a sortable value to use in place of the item . It can return anything sortable (a number, string, etc.). The items in the list will be sorted according to how their correponding return values from the key function would be sorted amongst each other.

Returning a tuple of sortable values is a convenient way to create a function that sorts on multiple fields. In this example, the user-defined class is sorted first by its name field and then, when there are any instances with the same name, by its value field.

Variable Capture

Besides a lambda expression’s arguments, a lambda expression can also access variables of the outer function that the lambda expression is contained within. This is called capture or closure.

For example:

In the above examples the captured variable is text . You can tell because it is a variable in the lambda expression that is not an argument (in these examples there are no arguments). The variable type is simply a string, but any data type including objects or other lambda functions can be captured:

If the variable holds an object, that object remains valid — it won’t be garbage-collected or destroyed — until the outer function exits and the lambda function is no longer referenced anywhere.

Lambda expressions are typically small, reusable, and self-contained, but capture makes lambda expressions less reusable and not self-contained, so excessive use of capture should be avoided.

How this works varies subtly across languages, and there are three types of variable capture that you need to be aware of.

Capture by Reference

Capture by reference means that the outer variable is shared with the lambda expression.

If the outer variable changes after the lambda expression is defined but before the lambda function is executed , the lambda function will get the updated value:

Conversely, the lambda expression can change the value of the outer variable so that when the lambda function finishes, the outer function sees the updated value:

Here’s a complex case. Can you figure out what will be printed?

In C#, Javascript, and Python, capture is always by reference.

In C++, there is no capture by default. To enable capture by reference, the & symbol can be put in the brackets that start the lambda expression. The brackets are for declaring capture.

When inside non-static methods of a class, the this variable is captured automatically in C#, Java, and Javascript and can be captured by value by adding this to the brackets in C++. That makes all of the current class instance’s fields, properties, methods, etc. available within the lambda expression as well.

In Python and Javascript, the capture rules apply to local function definitions the same as it does to lambda expressions. For example:

In Javascript, use the newer let keyword rather than var to declare variables to avoid for-loop scope mistakes.

Capture by Copy (C++)

Capture by copy means the lambda function gets a copy of the value in the capture variable at the time the lambda expression is defined . It cannot modify the outer variable and does not see changes to the outer variable.

In C++, capture can be either by reference or by copy. Using = in the capture brackets, capture is by copy.

Because a copy is made at the point where the lambda expression is declared, it will not see subsequent changes to the variable:

Variables captured by copy are const inside the lambda expression to prevent confusion about which variable is being edited:

Capture by copy is less prone to coding mistakes than capture by reference so it should be preferred, but it may come at a cost if the variable holds a complex data type that is expensive to copy.

Capture by copy should generally be used when capturing std::shared_ptr or other smart pointers because the copy will ensure the target object is not destroyed before the lambda function finishes. If that’s not a concern, capture by reference may be more efficient.

(In C#, Java, Javascript, and Python capture is always by reference (or by value in Java), so this section on capture by copy does not apply to those languages.)

Caveats and Capture by Value

It can become very difficult to track what captured variables will hold inside lambda expressions when the variable’s value changes, like in some of the examples in the previous section! It is especially hard in some specific cases:

• When the variable’s value changes between the lambda expression’s definition and its invocation.
• Capturing for -loop variables and variables within loops because their value may change on every iteration — in fact, they may be considered different variables on each iteration, which may or may not be what you expect.
• When lambda functions are executed asynchronously, as is often the case in Javascript, because the order of execution is opposite to the source code order and may be unpredictable.

Avoid using capture by reference in these circumstances whenever possible. especially capturing for -loop variables and variables within loops.

Modifying the captured variable either inside or outside the lambda expression is not possible in Java, which requires that captured variables be final or assigned once, so that it only ever has one value. That’s nice! It prevents complex situations that are prone to error. Since the captured variable cannot be changed, capture by reference is probably not the right term: capture by value might be a more appropriate term.

To get around these issues in all of the languages, you can sometimes make the captured variable a container (e.g. a list) and modify what is inside the container. Although the variable should not (or in Java cannot) be assigned a new value, its methods can be called and its fields and properties can be modified freely.

In addition to the captured variable’s value, it is can also be very hard to track its lifetime . Objects stored in captured variables may remain in memory so long as the lambda function remains stored in a variable. The lifetime of captured variables is dependent on the lifetime of the lambda function. If the lambda function is stored in a global variable, for example, any objects in variables captured by the lambda expression may hold onto system resources indefinitely.

Or, worse, those captured objects may become invalid in some way (due to their own semantics), leading to a crash or error the next time the lambda function is called. Lambda functions must be careful that nothing intervenes between definition and execution that would violate expectations the function has about the captured variables.

Capturing loop variables

The type of lambda expressions.

If you are writing your own functions that take lambda functions as arguments in statically typed languages, or if you are using C# ≤9 or Java and want to assign a lambda expression to a variable, you will need to know the correct types to use.

The type for lambda functions in C++ is std::function<return_type(arg_type_1, arg_type_2, ...)> which is defined in the functional standard header.

The template argument return_type(arg_type_1, arg_type_2, ...) is a little unusual, but it makes sense and pretty clearly indicates what’s what.

Three types in C# are generally used with lambda expressions.

System.Action is the usual type for lambda functions that do not return anything (like a void return type function). Without any generic arguments, it is the type for lambda functions that take no arguments and have no return value.

When System.Action has generic arguments, they are the lambda function’s argument types. For example, Action<string> is the type of a lambda function that takes a string and performs an action without returning a value. Action<string,Object> is the type of a lambda function that takes a string and an object as arguments (in that order) and performs an action without returning a value.

System.Func is the usual type for lambda functions that have a return value. The first (zero or more) generic arguments are the argument types. The last generic argument is always the type of the return value of the lambda function. So, for example, Func<string> is the type of a lambda function that takes no arguments and returns a string. Func<Object,string,int> is the type of a lambda function that takes an object and a string as arguments (in that order) and returns integer.

System.Predicate is a special case of Func where the return type is bool . So, for example, Predicate<string> is the type of a lambda function that takes a string argument and returns a boolean. Predicate<Object,string,int> is the type of a lambda function that takes those three argument types and returns a bool .

These types are used throughout the built-in .NET assemblies.

Actually any delegate type can be the type of a lambda expression, but there is generally no practical reason to use any type other than the three above unless the length of the names of these types becomes too cumbersome to type or read in the source code.

The var keyword can only be used with lambda expressions starting in C# 10, and only for some lambda expressions. Otherwise, you must give a full type.

In Java, there is no fixed type that must be used with a lambda expression. The pre-defined interface classes in java.util.function ( reference ) define some commonly used types, such as Function<T,R> which can be assigned a lambda function that takes a T as an argument and returns an R -type value. The Runnable ( reference ) interface can be used for lambda expressions that take no arguments and return nothing.

Any interface with a single method (a functional interface type) can be the type of a lambda expression, so long as the method has the same return type and arguments as the lambda expression. See the language reference .

Unfortunately, to invoke the lambda function, you must know the name of the method in the particular functional interface that is being used. Consult the interface documentation for the name of the method used to invoke the lambda function.

Dynamically Typed Languages

Although dynamically typed languages like Javascript and Python still have types at runtime , the types are generally not specified in source code.

Metaprogramming

In C#, lambda expressions can be used for metaprogramming. See System.Linq.Expressions.Expression .

Language-Specific Notes

In C++, capture can also be explicit: The brackets can contain a comma-separated list of variables to capture. Variables not mentioned in the list are not captured and not available to the lambda expression. Capture is by copy by default, unless the variable name is preceded with & . The this variable can also be listed when in a non-static class member function, and it is always captured by reference. ( refreence )

Use _ as the name of lambda expression arguments to indicate they are not used, rather than dummy1, dummy2, etc. _ is a lambda discard parameter when it is the name of more than one argument.

Capture can be turned off to avoid accidental capture or if you want to have a variable the same name as a variable in the outer scope by placing the static keyword immediately before the lambda expression.

Lambda expressions can use async .

Nothing yet.

Javascript has a longer, older quasi-alternative syntax to lambda expressions (called function expressions or anonymous functions) that looks like this:

They are almost equivalent. If you don’t use this , arguments , yield , and a few other rarely used keywords, the two syntaxes are pretty much the same.

As noted previously, the capture rules apply to function expressions the same as they do with lambda expressions.

As noted earlier, the capture rules apply to local function definitions the same as they do with lambda expressions.

Because Python variable scope is determined by the nearest assignment, variables in the outer scope cannot be assigned to:

The text variable in the inner function is a different variable than the one in the outer funtion because there is an assignment to text in the inner function. A trick is to make the outer variable a contaier:

Although Python capture is always by reference, there is a trick for achieving capture by value: assigning default values to lambda expression arguments.

It’s a weird and neat trick.

• C++: Lambda expressions
• C#: Lambda expression
• Java: Lambda expressions
• Javascript: Arrow function expressions/definitions
• Python: Lambda expressions

Holy Python

Lesson 11: Python Lambda

• ESTIMATED TIME: 15 mins
• DIFFICULTY: Intermediate
• FUNCTIONS: n/a
• 9 EXERCISES

Lambda, also called the anonymous function, is a perfect way to handle mini function necessities.

It doesn’t require creating definition blocks or naming the function and it all happens in one line.

When you need a proper user defined function defining a proper user function with  def keyword works great.

But sometimes you just need to squeeze your function in another function, or apply it to a column in a data frame or to the elements of an iterable like a list.

The necessity for lambda arises from those situations where you just need a one time quick solution in a practical way.

Although lambda syntax can be slightly confusing in the beginning, it can be quite handy once you get a hang of it.

Advantages of lambda is generally it’s tidy, practical and compact and it gets the job done which is a defining a custom function and then using it.

Used Where?

• As a quick alternative to a regular user defined function.
• .apply() : Used with pandas to apply functions to data frames and series.

Syntax do(s)

1) Start your argument with keyword lambda 2) Then variable name followed by colon(:) 3) Last part is the statement i.e.: lambda x: x+2

Syntax don't(s)

1) You can't have multiple lines of lambda 2) Don't put the colon (:) right after the lambda statement. Lambda first, variable second, colon third and statement forth and last.

Function : N/A

This lesson doesn’t introduce a new function.

Example 1: Simple lambda

>>> a = lambda x: x+2 >>> print(a(1))

• So, lambda argument starts with lambda . It’s the special phrase that starts your process.
• Then a variable (usually x for convenience) followed with a colon(:)
• Simply type your statement after the colon(:), and you’re done.

Example 2: print with lambda

>>> a = lambda x: x*3 >>> print(a(33))

To simplify things let’s explain again for the fresh programmers:

Program starts with lambda , then variable name and colon(:), on the right side of the colon simply statement is made regarding what happens to the variable.

Example 3: lambda with logical expression

>>> a = lambda x: x%5==0 >>> print(a(555)) >>> print(a(222))

We’re using a bunch of different Python operators here. Operators are fundamental to any programming language and Python is no exception.

If you never had a chance to formally study operators,  it can be useful to take a look at this Python Operators lesson which has a complete list of Python operators.

Printing the function object

You might at some point encounter a strange output when you’re learning Python. This happens when you try to print function objects directly without  without calling it with any value.

What you get will be the id of your function as a memory address rather than any meaningful value to the user.

Example 4: Printing the function

>>> a = lambda x: “Hi” >>> print(a)

at 0x00000163154BB1E0>

But what is the last printed line that looks weird?

This is because we are trying to print the function itself directly without calling it for any values. What happens is, Python prints the unique memory address of our function. That’s why it looks slightly weird. But it grows on you 🙂

Example 5: To print or not to print

>>> a = lambda x: “Hi” >>> a(2)

Example 6: lambda with two arguments

>>> a = lambda x, y: x*y >>> a(11, 6)

Tips (optional)

1- One of the most useful aspect of lambda is the ability to pass it as an argument inside some other functions.

We will look at each of these functions in detail in next lessons individually so you don’t have to ponder too much.

But it might be beneficial to take a look at them here to get familiar in advance.

Example 7: .sort() with lambda

.sort() is a list method that sorts the list’s elements directly.

Also it can take a function as an argument and base its sorting algorithm on this.

Simply pass your lambda function as an argument inside .sort() ‘s parenthesis.

>>> lst = [1, 5, 66, 7] >>> lst.sort(key=lambda x: x) print(lst)

[1, 5, 7, 66]

Example 8: sorted() with lambda

sorted() is not a method but a builtin function. It’s main difference from .sort() method is that it won’t change the original list it will simply output a new list. You have to assign this new list to a variable if you’d like to save it.

>>> lst = [1, 5, 66, 7] >>> lst_sorted = sorted(lst, key=lambda x: x)

Again you can see lambda is passed with “key=”.

Another syntactical difference from .sort() is that, since you’re not calling sorted() on a list directly you have to pass the list’s name inside its parenthesis before the “key=” parameter.

Example 9: map() with lambda

map() is a useful function that maps all the list items to a desired output. You can also check out our Python map lesson .

For example this example adds +5 to each item in the list.

This time, unlike sorted() , lambda comes first inside the parenthesis followed by comma and name of the list.

>>> lst = [1, 5, 66, 7] >>> lst_mapped = map(lambda x: x+5, lst) >>> print(list(lst_mapped))

[6, 10, 71, 12]

Example 10: filter() with lambda

Another function that works well with lambda , filter() is also similar to the map() function syntax wise. You can also check out our  Python filter lesson .

Inside parenthesis, type your lambda function followed by comma, and then the list you’d like to work on.

The type of functions lambda would implement inside filter function are usually logical expressions since the list will be filtered based on this logic. Let’s see the example:

>>> lst = [1, 5, 66, 7] >>> lst_filtered = filter(lambda x: x>5, lst) >>> print(list(lst_filtered))

Another point is that you actually don’t have to call your function inside a print() function when your function already has print in it.

Just call your function and it will do the printing for you.

Advanced Concepts (Optional)

1- You can even include an if statement in your lambda although its stretching the practical mini use intentions of lambda, syntactically it’s correct and you may find useful applications for it.

Let’s see an example.

Example 11: Lambda with if

if statement in lambda

>>> a = lambda x: print(“Aloha!”) if x==”Hawaii” else print(“Ciao!”) >>> print(“Hawaii”) >>> print(“London”)

Aloha! Ciao!

Example 12: Lambda with nested if

You can even have a nested if inside a lambda although it’s not a common use of functions and conditional statements at all.

>>> a = lambda x: print(“Aloha!”) if x==”Hawaii” else (print(“Salut”)  if x==”Paris” else print(“Hi”)

>>> print(“Hawaii”) >>> print(“Paris”) >>> print(“London”)

Aloha! Salut Hi

Example 13: Month names

Here is a cool example that can be used to change data from integers to month names. This can be quite relevant for specific real world applications.

Imagine you have a list of integers that stand for months. Using   map  and  lambda  and   calendar  library you can map these integers to month names.

calender.month_name[1] will return “ January “

>>> lst = lst=[1,2,3,4,5,6,7,4,3,2,5,11] >>> lst = map(lambda x: calendar.month_name[x], lst) >>> print(list(lst))

[‘January’, ‘February’, ‘March’, ‘April’, ‘May’, ‘June’, ‘July’, ‘April’, ‘March’, ‘February’, ‘May’, ‘November’]

Tip: You can do the same with day names using   day_name   function from calendar library.

calender.day_name[0] will return “ Monday “

We hope you liked this  lambda lesson.

Lambda is very practical and for this reason it’s heavily utilized in a number of key industries such as: finance, data science, data visualization, scripting and cyber security as well as other fields.

Do you think you got the hang of lambda? We have online interactive lambda exercises in Python that might help you test your skills or master the concept!

9 Exercises About Lambda functions in Python

Next Lesson: zip() function

• Documentation »
• Correctness »
• Assigning a lambda expression to a variable
• View page source

Assigning a lambda expression to a variable ¶

The sole advantage that a lambda expression has over a def is that the lambda can be anonymously embedded within a larger expression. If you are going to assign a name to a lambda , you are better off just defining it as a def .

From the PEP 8 Style Guide:

The first form means that the name of the resulting function object is specifically ‘f’ instead of the generic ‘<lambda>’. This is more useful for tracebacks and string representations in general. The use of the assignment statement eliminates the sole benefit a lambda expression can offer over an explicit def statement (i.e. that it can be embedded inside a larger expression)

Anti-pattern ¶

The following code assigns a lambda function which returns the double of its input to a variable. This is functionally identical to creating a def .

Best practice ¶

Use a def for named expressions ¶.

Refactor the lambda expression into a named def expression.

References ¶

• PEP 8 Style Guide - Programming Recommendations
• Stack Overflow - Do not assign a lambda expression

This browser is no longer supported.

Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.

Lambda expressions and anonymous functions

• 6 contributors

You use a lambda expression to create an anonymous function. Use the lambda declaration operator => to separate the lambda's parameter list from its body. A lambda expression can be of any of the following two forms:

Expression lambda that has an expression as its body:

Statement lambda that has a statement block as its body:

To create a lambda expression, you specify input parameters (if any) on the left side of the lambda operator and an expression or a statement block on the other side.

Any lambda expression can be converted to a delegate type. The delegate type to which a lambda expression can be converted is defined by the types of its parameters and return value. If a lambda expression doesn't return a value, it can be converted to one of the Action delegate types; otherwise, it can be converted to one of the Func delegate types. For example, a lambda expression that has two parameters and returns no value can be converted to an Action<T1,T2> delegate. A lambda expression that has one parameter and returns a value can be converted to a Func<T,TResult> delegate. In the following example, the lambda expression x => x * x , which specifies a parameter that's named x and returns the value of x squared, is assigned to a variable of a delegate type:

Expression lambdas can also be converted to the expression tree types, as the following example shows:

You can use lambda expressions in any code that requires instances of delegate types or expression trees, for example as an argument to the Task.Run(Action) method to pass the code that should be executed in the background. You can also use lambda expressions when you write LINQ in C# , as the following example shows:

When you use method-based syntax to call the Enumerable.Select method in the System.Linq.Enumerable class, for example in LINQ to Objects and LINQ to XML, the parameter is a delegate type System.Func<T,TResult> . When you call the Queryable.Select method in the System.Linq.Queryable class, for example in LINQ to SQL, the parameter type is an expression tree type Expression<Func<TSource,TResult>> . In both cases, you can use the same lambda expression to specify the parameter value. That makes the two Select calls to look similar although in fact the type of objects created from the lambdas is different.

Expression lambdas

A lambda expression with an expression on the right side of the => operator is called an expression lambda . An expression lambda returns the result of the expression and takes the following basic form:

The body of an expression lambda can consist of a method call. However, if you're creating expression trees that are evaluated outside the context of the .NET Common Language Runtime (CLR), such as in SQL Server, you shouldn't use method calls in lambda expressions. The methods have no meaning outside the context of the .NET Common Language Runtime (CLR).

Statement lambdas

A statement lambda resembles an expression lambda except that its statements are enclosed in braces:

The body of a statement lambda can consist of any number of statements; however, in practice there are typically no more than two or three.

You can't use statement lambdas to create expression trees.

Input parameters of a lambda expression

You enclose input parameters of a lambda expression in parentheses. Specify zero input parameters with empty parentheses:

If a lambda expression has only one input parameter, parentheses are optional:

Two or more input parameters are separated by commas:

Sometimes the compiler can't infer the types of input parameters. You can specify the types explicitly as shown in the following example:

Input parameter types must be all explicit or all implicit; otherwise, a CS0748 compiler error occurs.

You can use discards to specify two or more input parameters of a lambda expression that aren't used in the expression:

Lambda discard parameters may be useful when you use a lambda expression to provide an event handler .

For backwards compatibility, if only a single input parameter is named _ , then, within a lambda expression, _ is treated as the name of that parameter.

Beginning with C# 12, you can provide default values for parameters on lambda expressions. The syntax and the restrictions on default parameter values are the same as for methods and local functions. The following example declares a lambda expression with a default parameter, then calls it once using the default and once with two explicit parameters:

You can also declare lambda expressions with params arrays as parameters:

As part of these updates, when a method group that has a default parameter is assigned to a lambda expression, that lambda expression also has the same default parameter. A method group with a params array parameter can also be assigned to a lambda expression.

Lambda expressions with default parameters or params arrays as parameters don't have natural types that correspond to Func<> or Action<> types. However, you can define delegate types that include default parameter values:

Or, you can use implicitly typed variables with var declarations to define the delegate type. The compiler synthesizes the correct delegate type.

For more information on see the feature spec for default parameters on lambda expressions .

Async lambdas

You can easily create lambda expressions and statements that incorporate asynchronous processing by using the async and await keywords. For example, the following Windows Forms example contains an event handler that calls and awaits an async method, ExampleMethodAsync .

You can add the same event handler by using an async lambda. To add this handler, add an async modifier before the lambda parameter list, as the following example shows:

For more information about how to create and use async methods, see Asynchronous Programming with async and await .

Lambda expressions and tuples

The C# language provides built-in support for tuples . You can provide a tuple as an argument to a lambda expression, and your lambda expression can also return a tuple. In some cases, the C# compiler uses type inference to determine the types of tuple components.

You define a tuple by enclosing a comma-delimited list of its components in parentheses. The following example uses tuple with three components to pass a sequence of numbers to a lambda expression, which doubles each value and returns a tuple with three components that contains the result of the multiplications.

Ordinarily, the fields of a tuple are named Item1 , Item2 , and so on. You can, however, define a tuple with named components, as the following example does.

For more information about C# tuples, see Tuple types .

Lambdas with the standard query operators

LINQ to Objects, among other implementations, has an input parameter whose type is one of the Func<TResult> family of generic delegates. These delegates use type parameters to define the number and type of input parameters, and the return type of the delegate. Func delegates are useful for encapsulating user-defined expressions that are applied to each element in a set of source data. For example, consider the Func<T,TResult> delegate type:

The delegate can be instantiated as a Func<int, bool> instance where int is an input parameter and bool is the return value. The return value is always specified in the last type parameter. For example, Func<int, string, bool> defines a delegate with two input parameters, int and string , and a return type of bool . The following Func delegate, when it's invoked, returns Boolean value that indicates whether the input parameter is equal to five:

You can also supply a lambda expression when the argument type is an Expression<TDelegate> , for example in the standard query operators that are defined in the Queryable type. When you specify an Expression<TDelegate> argument, the lambda is compiled to an expression tree.

The following example uses the Count standard query operator:

The compiler can infer the type of the input parameter, or you can also specify it explicitly. This particular lambda expression counts those integers ( n ) which when divided by two have a remainder of 1.

The following example produces a sequence that contains all elements in the numbers array that precede the 9, because that's the first number in the sequence that doesn't meet the condition:

The following example specifies multiple input parameters by enclosing them in parentheses. The method returns all the elements in the numbers array until it finds a number whose value is less than its ordinal position in the array:

You don't use lambda expressions directly in query expressions , but you can use them in method calls within query expressions, as the following example shows:

Type inference in lambda expressions

When writing lambdas, you often don't have to specify a type for the input parameters because the compiler can infer the type based on the lambda body, the parameter types, and other factors as described in the C# language specification. For most of the standard query operators, the first input is the type of the elements in the source sequence. If you're querying an IEnumerable<Customer> , then the input variable is inferred to be a Customer object, which means you have access to its methods and properties:

The general rules for type inference for lambdas are as follows:

• The lambda must contain the same number of parameters as the delegate type.
• Each input parameter in the lambda must be implicitly convertible to its corresponding delegate parameter.
• The return value of the lambda (if any) must be implicitly convertible to the delegate's return type.

Natural type of a lambda expression

A lambda expression in itself doesn't have a type because the common type system has no intrinsic concept of "lambda expression." However, it's sometimes convenient to speak informally of the "type" of a lambda expression. That informal "type" refers to the delegate type or Expression type to which the lambda expression is converted.

Beginning with C# 10, a lambda expression may have a natural type . Instead of forcing you to declare a delegate type, such as Func<...> or Action<...> for a lambda expression, the compiler may infer the delegate type from the lambda expression. For example, consider the following declaration:

The compiler can infer parse to be a Func<string, int> . The compiler chooses an available Func or Action delegate, if a suitable one exists. Otherwise, it synthesizes a delegate type. For example, the delegate type is synthesized if the lambda expression has ref parameters. When a lambda expression has a natural type, it can be assigned to a less explicit type, such as System.Object or System.Delegate :

Method groups (that is, method names without parameter lists) with exactly one overload have a natural type:

If you assign a lambda expression to System.Linq.Expressions.LambdaExpression , or System.Linq.Expressions.Expression , and the lambda has a natural delegate type, the expression has a natural type of System.Linq.Expressions.Expression<TDelegate> , with the natural delegate type used as the argument for the type parameter:

Not all lambda expressions have a natural type. Consider the following declaration:

The compiler can't infer a parameter type for s . When the compiler can't infer a natural type, you must declare the type:

Explicit return type

Typically, the return type of a lambda expression is obvious and inferred. For some expressions that doesn't work:

Beginning with C# 10, you can specify the return type of a lambda expression before the input parameters. When you specify an explicit return type, you must parenthesize the input parameters:

Beginning with C# 10, you can add attributes to a lambda expression and its parameters. The following example shows how to add attributes to a lambda expression:

You can also add attributes to the input parameters or return value, as the following example shows:

As the preceding examples show, you must parenthesize the input parameters when you add attributes to a lambda expression or its parameters.

Lambda expressions are invoked through the underlying delegate type. That is different than methods and local functions. The delegate's Invoke method doesn't check attributes on the lambda expression. Attributes don't have any effect when the lambda expression is invoked. Attributes on lambda expressions are useful for code analysis, and can be discovered via reflection. One consequence of this decision is that the System.Diagnostics.ConditionalAttribute cannot be applied to a lambda expression.

Capture of outer variables and variable scope in lambda expressions

Lambdas can refer to outer variables . These outer variables are the variables that are in scope in the method that defines the lambda expression, or in scope in the type that contains the lambda expression. Variables that are captured in this manner are stored for use in the lambda expression even if the variables would otherwise go out of scope and be garbage collected. An outer variable must be definitely assigned before it can be consumed in a lambda expression. The following example demonstrates these rules:

The following rules apply to variable scope in lambda expressions:

• A variable that is captured won't be garbage-collected until the delegate that references it becomes eligible for garbage collection.
• Variables introduced within a lambda expression aren't visible in the enclosing method.
• A lambda expression can't directly capture an in , ref , or out parameter from the enclosing method.
• A return statement in a lambda expression doesn't cause the enclosing method to return.
• A lambda expression can't contain a goto , break , or continue statement if the target of that jump statement is outside the lambda expression block. It's also an error to have a jump statement outside the lambda expression block if the target is inside the block.

You can apply the static modifier to a lambda expression to prevent unintentional capture of local variables or instance state by the lambda:

A static lambda can't capture local variables or instance state from enclosing scopes, but may reference static members and constant definitions.

C# language specification

For more information, see the Anonymous function expressions section of the C# language specification .

For more information about these features, see the following feature proposal notes:

• Lambda discard parameters
• Static anonymous functions
• Lambda improvements (C# 10)
• Use local function instead of lambda (style rule IDE0039)
• C# operators and expressions
• LINQ (Language-Integrated Query)
• Expression trees
• Local functions vs. lambda expressions
• LINQ sample queries
• XQuery sample
• 101 LINQ samples

Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback .

Submit and view feedback for

AWS Compute Blog

Automating chaos experiments with aws fault injection service and aws lambda.

This post is written by André Stoll, Solution Architect.

Chaos engineering is a popular practice for building confidence in system resilience. However, many existing tools assume the ability to alter infrastructure configurations, and cannot be easily applied to the serverless application paradigm. Due to the stateless, ephemeral, and distributed nature of serverless architectures, you must evolve the traditional technique when running chaos experiments on these systems.

This blog post explains a technique for running chaos engineering experiments on AWS Lambda functions. The approach uses Lambda extensions to induce failures in a runtime-agnostic way requiring no function code changes. It shows how you can use the AWS Fault Injection Service (FIS) to automate and manage chaos experiments across different Lambda functions to provide a reusable testing method.

Chaos experiments are commonly applied to cloud applications to uncover latent issues and prevent service disruptions. IT teams use chaos experiments to build confidence in the robustness of their systems. However, the traditional methods used in server-based chaos engineering do not easily translate to the serverless world since many existing tools are based on altering the underlying infrastructure configurations, such as cluster nodes or server instances of your applications.

In serverless applications, AWS handles the undifferentiated heavy lifting of managing infrastructure, so you can focus on delivering business value. But this also means that engineering teams have limited control over the infrastructure, and must rely on application-level tooling to run chaos experiments. Two techniques commonly used in the serverless community for conducting chaos experiments on Lambda functions are modifying the function configuration or using runtime-specific libraries.

Changing the configuration of a Lambda function allows you to induce rudimentary failures. For example, you can set the reserved concurrency of a Lambda function to simulate invocation throttling. Alternatively, you might change the function execution role permissions or the function policy to simulate IAM access denial. These types of failures are easy to implement, but the range of possible fault injection types is limited.

The other technique—injecting chaos into Lambda functions through purpose-built, runtime-specific libraries—is more flexible. There are various open-source libraries that allow you to inject failures, such as added latency, exceptions, or disk exhaustion. Examples of such libraries are Python’s chaos_lambda and failure-lambda for Node.js. The downside is that you must change the function code for every function you want to run chaos experiments on. In addition, those libraries are runtime-specific and each library comes with a set of different capabilities and configurations. This reduces the reusability of your chaos experiments across Lambda functions implemented in different languages.

Injecting chaos using Lambda extensions

Implementing chaos experiments using Lambda extensions allows you to address all of the previous concerns. Lambda extensions augment your functions by adding functionality, such as capturing diagnostic information or automatically instrumenting your code. You can integrate your preferred monitoring, observability, or security tooling deeply into the Lambda environment without complex installation or configuration management. Lambda extensions are generally packaged as Lambda layers and run as a separate process in the Lambda execution environment . You may use extensions from AWS , AWS Lambda partners , or build your own custom functionality.

With Lambda extensions, you can implement a chaos extension to inject the desired failures into your Lambda environments. This chaos extension uses the Runtime API proxy pattern that enables you to hook into the function invocation request and response lifecycle. Lambda runtimes use the Lambda Runtime API to retrieve the next incoming event to be processed by the function handler and return the handler response to the Lambda service.

The Runtime API HTTP endpoint is available within the Lambda execution environment. Runtimes get the API endpoint from the environment variable AWS_LAMBDA_RUNTIME_API . During the initialization of the execution environment, you can modify the runtime startup behavior. This lets you change the value of AWS_LAMBDA_RUNTIME_API to the port the chaos extension process is listening on. Now, all requests to the Runtime API go through the chaos extension proxy. You can use this workflow for blocking malicious events, auditing payloads, or injecting failures.

• The chaos extension intercepts incoming events and outbound responses, and injects failures according to the chaos experiment configuration.
• The extension accesses environment variables to read the chaos experiment configuration.
• A wrapper script configures the runtime to proxy requests through the chaos extension.

When intercepting incoming events and outbound responses to the Lambda Runtime API, you can simulate failures such as introducing artificial delay or generate an error response to return to the Lambda service. This workflow adds latency to your function calls:

All Lambda runtimes support extensions. Since extensions run as a separate process, you can implement them in a language other than the function code. AWS recommends you implement extensions using a programming language that compiles to a binary executable, such as Golang or Rust. This allows you to use the extension with any Lambda runtime.

Some of the open source projects following this technique are the chaos-lambda-extension , implemented in Rust, or the serverless-chaos-extension , implemented in Python.

Extensions provide you with a flexible and reusable method to run your chaos experiments on Lambda functions. You can reuse the chaos extension for all runtimes without having to change function code. Add the extension to any Lambda function where you want to run chaos experiments.

Automating with AWS FIS experiment templates

According to the Principles of Chaos Engineering , you should “automate your experiments to run continuously”. To achieve this, you can use the AWS Fault Injection Service (FIS).

This service allows you to generate reusable experiment templates . The template specifies the targets and the actions to run on them during the experiment, and an optional stop condition that prevents the experiment from going out of bounds. You can also execute AWS Systems Manager Automation runbooks which support custom fault types. You can write your own custom Systems Manager documents to define the individual steps involved in the automation. To carry out the actions of the experiment, you define scripts in the document to manage your Lambda function and set it up for the chaos experiment.

To use the chaos extension for your serverless chaos experiments:

• Set up the Lambda function for the experiment. Add the chaos extension as a layer and configure the experiment, for example, by adding environment variables specifying the fault type and its corresponding value.
• Pause the automation and conduct the experiment. To do this, use the aws:sleep automation action. During this period, you conduct the experiment, measure and observe the outcome.
• Clean up the experiment. The script removes the layer again and also resets the environment variables.

Running your first serverless chaos experiment

This sample repository provides you with the necessary code to run your first serverless chaos experiment in AWS. The experiment uses the chaos-lambda-extension extension to inject chaos.

The sample deploys the AWS FIS experiment template, the necessary SSM Automation runbooks including the IAM role used by the runbook to configure the Lambda functions. The sample also provisions a Lambda function for testing and an Amazon CloudWatch alarm used to roll back the experiment.

Prerequisites

• You have the chaos extension already deployed as a Lambda layer in your AWS account.
• You have installed the AWS Cloud Development Kit (CDK) CLI. See the Getting started with the AWS CDK guide for details.
• Clone the sample repository locally by running git clone [email protected]:aws-samples/serverless-chaos-experiments.git
• Ensure you have sufficient permissions to interact with the AWS FIS, Lambda, and CloudWatch alarms.

Running the experiment

Follow the steps outlined in the repository to conduct your first experiment. Starting the experiment triggers the automation execution.

This automation includes adding the extension and configuring the experiment, pausing the execution and observing the system and reverting all changes to the initial state.

If you invoke the targeted Lambda function during the second step, failures (in this case, artificial latency) are simulated.

Security best practices

Extensions run within the same execution environment as the function, so they have the same level of access to resources such as file system, networking, and environment variables. IAM permissions assigned to the function are shared with extensions. AWS recommends you assign the least required privileges to your functions.

Always install extensions from a trusted source only. Use Infrastructure as Code (IaC) and automation tools, such as CloudFormation or AWS Systems Manager , to simplify attaching the same extension configuration, including AWS Identity and Access Management (IAM) permissions, to multiple functions. IaC and automation tools allow you to have an audit record of extensions and versions used previously.

When building extensions, do not log sensitive data. Sanitize payloads and metadata before logging or persisting them for audit purposes.

This blog post details how to run chaos experiments for serverless applications built using Lambda. The described approach uses Lambda extension to inject faults into the execution environment. This allows you to use the same method regardless of runtime or configuration of the Lambda function.

To automate and successfully conduct the experiment, you can use the AWS Fault Injection Service. By creating an experiment template, you can specify the actions to run on the defined targets, such as adding the extension during the experiment. Since the extension can be used for any runtime, you can reuse the experiment template to inject failures into different Lambda functions.

Visit this repository to deploy your first serverless chaos experiment, or watch this video guide for learning more about building extensions. Explore the AWS FIS documentation to learn how to create your own experiments.

For more serverless learning resources, visit Serverless Land .