Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 15 additions & 84 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,92 +1,23 @@
# r2lambda 0.3.0

## New features
* First CRAN release.

* raise errors with cli ([#49](https://github.com/discindo/r2lambda/issues/49)) ([b72e349](https://github.com/discindo/r2lambda/commit/b72e349f9fdb86413910bd8dda842cf41ce83875))
* `build_lambda()` builds a Docker image for an R runtime script, with R
packages from a list or an `renv.lock` file, Amazon Linux 2023 system
packages, support files and folders, and `x86_64` or `arm64`
architecture.

* `test_lambda()` runs the image in a local container and invokes it.

## Bug fixes
* `deploy_lambda()` pushes the image to the AWS Elastic Container Registry
and creates the Lambda function, with a new execution role or an
existing one.

* check deploy_lambda() inputs before pushing the image ([#59](https://github.com/discindo/r2lambda/issues/59)) ([691a9f8](https://github.com/discindo/r2lambda/commit/691a9f80e967a1bde7b083c07608303d534b6bd4))
* keep schedule rule and statement names distinct and within limits ([#56](https://github.com/discindo/r2lambda/issues/56)) ([13c8dc3](https://github.com/discindo/r2lambda/commit/13c8dc3f6a239a4d8d3010e5aedb400c25141b8d))
* limit delete_lambda() to the current region's unused roles ([#57](https://github.com/discindo/r2lambda/issues/57)) ([aa17fe9](https://github.com/discindo/r2lambda/commit/aa17fe93411615433c53688cfaaea7b0583d4919))
* restore renv lockfiles from their own repositories ([#58](https://github.com/discindo/r2lambda/issues/58)) ([2d9b944](https://github.com/discindo/r2lambda/commit/2d9b944477e9616b460bb024ea160d4f5cd43430))
* retry test_lambda() requests on any curl error ([#55](https://github.com/discindo/r2lambda/issues/55)) ([2cc00f5](https://github.com/discindo/r2lambda/commit/2cc00f542281c8b6611543f821f39250e33469c8))
* `invoke_lambda()` invokes a deployed function, `schedule_lambda()` runs
it on an EventBridge schedule, and `delete_lambda()` removes its AWS
resources and local image.

# r2lambda 0.2.0

## New features

* add delete_lambda() to remove a function's AWS resources ([#41](https://github.com/discindo/r2lambda/issues/41)) ([13abc85](https://github.com/discindo/r2lambda/commit/13abc855b020902f7f14a916fcd34ce72f7d40d8))


## Bug fixes

* keep IAM role names within 64 characters ([#46](https://github.com/discindo/r2lambda/issues/46)) ([f21a0bd](https://github.com/discindo/r2lambda/commit/f21a0bde854faaaf0ed48af2c8dd41cc044a7e18))
* send EventBridge target ID as a string ([#40](https://github.com/discindo/r2lambda/issues/40)) ([d69201a](https://github.com/discindo/r2lambda/commit/d69201a6c0f43d3e609556c16198e83e84e9cd81))


## Documentation

* convert vignettes to Quarto ([#37](https://github.com/discindo/r2lambda/issues/37)) ([2fd68c7](https://github.com/discindo/r2lambda/commit/2fd68c7d1e9506929859dd1596c6b0d48063952c))
* group articles under a Tutorials heading ([#45](https://github.com/discindo/r2lambda/issues/45)) ([624fbbd](https://github.com/discindo/r2lambda/commit/624fbbd60ec7c0893f2960c33b3ab4a7b34f224b))
* maintain README.md directly and drop README.Rmd ([#39](https://github.com/discindo/r2lambda/issues/39)) ([3b2ff98](https://github.com/discindo/r2lambda/commit/3b2ff98a52bc36b4f195308ea18a053a29876ccf))
* meet CRAN extra checks ([#42](https://github.com/discindo/r2lambda/issues/42)) ([4968dbf](https://github.com/discindo/r2lambda/commit/4968dbf65a62545b6cf2ff92a87207ac62df9630))
* move vignettes to pkgdown-only articles ([#43](https://github.com/discindo/r2lambda/issues/43)) ([c958ce6](https://github.com/discindo/r2lambda/commit/c958ce6538438f136b1961fba66bb21709667929))

# r2lambda 0.1.0

## New features

* accept folders in support_path ([27254c3](https://github.com/discindo/r2lambda/commit/33725e644636c1a8be2c10793192dce5b6e53308))
* add renv logic when creating the dockerfile ([5e94146](https://github.com/discindo/r2lambda/commit/a7d42fff132825c7c4af16eebbcc05ab705969c2))
* allow support files (multi-file project) ([8180779](https://github.com/discindo/r2lambda/commit/5fd71e166887344d66ec9e5d6c19c38b8b49058a))
* allow support files (multi-file project) ([95f9949](https://github.com/discindo/r2lambda/commit/44379b375e7961912ead279293ebde343549df06))
* build and deploy arm64 Lambda functions ([d9b3bde](https://github.com/discindo/r2lambda/commit/c185b116a317dad6fbaa25e67e2aef642fb51c2b))
* check lambda state before invoking ([bf37034](https://github.com/discindo/r2lambda/commit/bf37034d91204f71a33887ba13d2d7c637d700f5))
* deploy with an existing IAM role ([8cfe969](https://github.com/discindo/r2lambda/commit/fc25d4d0b35e6969d87f97d5689caddfc729d0c1)), closes [#6](https://github.com/discindo/r2lambda/issues/6)
* grant AWS access through the Lambda execution role ([1ec810a](https://github.com/discindo/r2lambda/commit/0537753e829102e4e2b2b4581dbbc5855a608f6d))
* install system packages in the Lambda image ([3d642a6](https://github.com/discindo/r2lambda/commit/256e40f01938cda26531868436b4ebb46bfbe8b3)), closes [#5](https://github.com/discindo/r2lambda/issues/5)
* parse ecr token. ([9adf88a](https://github.com/discindo/r2lambda/commit/9adf88a18f1603e0378b6c674f345fe2e245f4bc))
* separate build from deploy ([e2847ca](https://github.com/discindo/r2lambda/commit/e2847cacf48b0a43652ae55cb7cb3ddd6aa7bc47))
* start, test, and stop a lambda container locally ([d252680](https://github.com/discindo/r2lambda/commit/97862a6718e409b985f6e1e01efb9124ffe4986b))
* start, test, and stop a lambda container locally ([dc213d2](https://github.com/discindo/r2lambda/commit/a499635b9a698009d4b54bbd3dfe22b7ed6f5333))
* try to catch failed invocation, including due to Inactive state ([881f329](https://github.com/discindo/r2lambda/commit/881f329d98b96ffb06a0304b98386554dfa99948))
* update lambda to run on schedule ([7f4472c](https://github.com/discindo/r2lambda/commit/456a6ab7ba173b179b01bfad0fcc8abffeb8b351))
* use magrittr pipe ([6fa8575](https://github.com/discindo/r2lambda/commit/6fa85759d5d60ee9f1d192e497eb8b1c75d81282))
* use stevedore for push to ecr ([3e6641a](https://github.com/discindo/r2lambda/commit/3e6641a1cdf5db235a1d35969833e77582d2eab3))
* use stevedore to create the docker image ([a12a491](https://github.com/discindo/r2lambda/commit/a12a491e987cb256e5e257f704f8dc096af9e83c))


## Bug fixes

* build Lambda images on Amazon Linux 2023 with R 4.6.1 ([d573317](https://github.com/discindo/r2lambda/commit/161a5f9fc76a09462490d1461b4f5ef57faf261e))
* call the docker CLI instead of stevedore ([c711055](https://github.com/discindo/r2lambda/commit/35b96e65825f84c768eae040a89a90500e52f63f)), closes [#11](https://github.com/discindo/r2lambda/issues/11)
* check the image architecture before pushing it ([d74aaf3](https://github.com/discindo/r2lambda/commit/03f7703647fc60238eeead7c6ea8bfbc80b89e7b))
* correct log msg ([79a3df0](https://github.com/discindo/r2lambda/commit/79a3df0cce63ea9f535102c8d701eceb7a6eac49))
* don't create an ECR repository when deploying ([a669cef](https://github.com/discindo/r2lambda/commit/6d20439192488c1fb28f3232f1ddc5647e630069))
* don't need to check for aws cli anymore ([673b69c](https://github.com/discindo/r2lambda/commit/673b69c7720182b3c6bf87d16a5c963d2fd2f741))
* inspect the same image that deploy_lambda pushes ([ec91125](https://github.com/discindo/r2lambda/commit/26b635b9afd6c9900054b35a24a3a25d402c4e73))
* install from github in readme ([61e6435](https://github.com/discindo/r2lambda/commit/61e643567f470e9d556cc476c00ddddcfe900565))
* look up AWS resources by name instead of listing them ([a99bfc0](https://github.com/discindo/r2lambda/commit/5b7f38d4480e9ce7397365c59862cf655821228d))
* name support files in the Dockerfile as they are copied ([bbf5296](https://github.com/discindo/r2lambda/commit/0766ac1f3d1f63edf80c1edc7e787d935c6fe8dc))
* report AWS errors from invoke_lambda and deploy_lambda ([1ceef40](https://github.com/discindo/r2lambda/commit/28912ffe69b2da60d51723ede8e989b8c23883f7))
* tempdir issue in tests ([28032d9](https://github.com/discindo/r2lambda/commit/ac5a18d58c113fcf01feee7469d92251a8b17a00))


## Documentation

* add Novica Nakov as maintainer and ORCIDs for both authors ([9c20ff7](https://github.com/discindo/r2lambda/commit/6b9b8a3c25ef238b5ff089e7b97d0ad707574aed))
* build ignore docs folder ([8a5af02](https://github.com/discindo/r2lambda/commit/8a5af02af3cd512f79815d8c15e1a64d8992bd7e))
* code of conduct. ([9608875](https://github.com/discindo/r2lambda/commit/96088754bc19dd9a7df75494b2cd4b730d0f4ed8))
* correction in readme ([55b077a](https://github.com/discindo/r2lambda/commit/55b077a32360e4fd2bc57df16edf61619492f80b))
* document role, policy, support folder, system package and architecture options ([e2a5d07](https://github.com/discindo/r2lambda/commit/058096088f9ee4282883852c08c70962ef901490))
* fix moved blog link in lambda-s3 vignette ([4471463](https://github.com/discindo/r2lambda/commit/8c06a4005b8c891b85ae044127470eea81686437))
* manual files ([16ba1e2](https://github.com/discindo/r2lambda/commit/16ba1e261a38186613853031b5096d90864be44d))
* manual files ([4c16601](https://github.com/discindo/r2lambda/commit/4c16601c2b3a4c23447efa52d148401f577d682b))
* manual files and vignette ([f470ef7](https://github.com/discindo/r2lambda/commit/815449e7a6e18a47b8d928e63675af99d26fab4b))
* pkg description ([9b778d1](https://github.com/discindo/r2lambda/commit/9b778d1f880d4ccf29ac6823b63de0c3ebdb6625))
* typo ([41386a6](https://github.com/discindo/r2lambda/commit/41386a64fb6c76979dff7df16fdde333fd420882))
* update readme ([a002958](https://github.com/discindo/r2lambda/commit/71927624c861f1f145aa3445a0b95055b26d4d9d))
* update readme ([f89c595](https://github.com/discindo/r2lambda/commit/f89c5957604c8a2ceec9334d959ca691d150add1))
* `aws_connect()` creates a `paws` client for an AWS service.

Changes made before the first CRAN release are listed in
[CHANGELOG.md](https://github.com/discindo/r2lambda/blob/main/CHANGELOG.md).
148 changes: 34 additions & 114 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,23 @@ coverage](https://codecov.io/gh/discindo/r2lambda/graph/badge.svg)](https://app.
The goal of `{r2lambda}` is to make it easier to go from an `R` script
to a deployed `AWS Lambda` function.

## Requirements
## System requirements

- [docker](https://docs.docker.com/get-docker/) with the
`{r2lambda}` calls two programs outside R. They are the same on Linux,
macOS and Windows; only how you install them differs.

- [Docker](https://docs.docker.com/get-docker/) with the
[buildx](https://docs.docker.com/build/concepts/overview/#buildx)
plugin is required to build, tag, and push the image. Docker Desktop
includes buildx; on Linux, some distributions package it separately,
e.g. `docker-buildx`.
- [curl](https://curl.se/) 7.71 or later is required by `test_lambda()`
to call the local container.
- An AWS account. `build_lambda()` already creates the ECR repository
for the image, and `deploy_lambda()` creates an IAM role and the
Lambda function.
plugin, to build, run and push images. On macOS and Windows, install
Docker Desktop, which includes buildx; on Windows, use its default
Linux containers. On Linux, install Docker Engine and the buildx
plugin, which some distributions package separately, e.g.
`docker-buildx`. The `docker` command must be on the `PATH` and the
Docker daemon must be running.
- [curl](https://curl.se/) 7.71 or later, used by `test_lambda()` to
call the local container. macOS and current Windows 10 and 11 include
it; on Linux, install it with your package manager. Check the version
with `curl --version`.

## Installation

Expand All @@ -40,6 +45,10 @@ remotes::install_github("discindo/r2lambda")

## Setup

You need an AWS account. `build_lambda()` creates the ECR repository
for the image, and `deploy_lambda()` creates an IAM role and the Lambda
function.

`r2lambda` assumes credentials for connecting to AWS services are
available in the `R` session. This can be done via an `.Renviron` file
that should set environmental variables like so:
Expand All @@ -54,130 +63,41 @@ methods supported by `paws` are available in `r2lambda`. See
[here](https://github.com/paws-r/paws/blob/main/docs/credentials.md) for
details on setting credentials, region, profile, etc.

## Workflow

### Build a docker image for the lambda function
## Example

The examples use two files installed with the package:
`parity.R`, a runtime script whose `parity()` function tells whether a
number is even or odd, and `renv.lock`, an example lockfile.
`system.file()` returns their paths. Replace them with your own script
and lockfile.
Build an image from `parity.R`, an example runtime script installed
with the package, test it locally, deploy it, invoke it, and delete it:

``` r
runtime_function <- "parity"
runtime_path <- system.file("parity.R", package = "r2lambda")
renvlock_path <- system.file("renv.lock", package = "r2lambda")
dependencies <- NULL
library(r2lambda)

# Might take a while, its building a docker image
build_lambda(
tag = "parity1",
runtime_function = runtime_function,
runtime_path = runtime_path,
renvlock_path = renvlock_path,
dependencies = dependencies
)
```

`build_lambda()` has a few more options:

- `support_path`: files or folders the runtime script needs. Each is
copied next to the runtime script under its own name, so
`support_path = "R"` lets the script call `source("R/helpers.R")`.
- `system_deps`: Amazon Linux 2023 packages to install with `dnf`, for R
packages that need system libraries, e.g. `system_deps = "udunits2"`
for `units`. Amazon Linux 2023 has no EPEL, so some libraries, such as
GDAL, are not available.
- `architecture`: `"x86_64"` (default) or `"arm64"`. `deploy_lambda()`
creates the function with the image’s architecture. Building for an
architecture other than your machine’s needs Docker’s QEMU emulation.

### Test the lambda docker image locally

``` r
payload <- list(number = 2)
tag <- "parity1"
test_lambda(tag = "parity1", payload)
```

### Deploy to AWS Lambda

``` r
# Might take a while, its pushing it to a remote repository
deploy_lambda(tag = "parity1")
```

Lambda stops a function after 3 seconds by default, which an R
container may not meet, especially on its first call. Pass a longer
`Timeout`, in seconds, and other `create_function()` settings through
`...`:

``` r
deploy_lambda(tag = "parity1", Timeout = 30, MemorySize = 512)
```

The function runs under a new IAM role with basic execution permissions.
To let it call other AWS services, attach policies to that role, or use
a role you already have:

``` r
deploy_lambda(
tag = "parity1",
policy_arns = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"
runtime_function = "parity",
runtime_path = system.file("parity.R", package = "r2lambda"),
renvlock_path = system.file("renv.lock", package = "r2lambda")
)

deploy_lambda(tag = "parity1", role_arn = "arn:aws:iam::123456789012:role/my-role")
```

Inside the function, `paws` picks up the role’s temporary credentials,
so no AWS keys go into the image or the function’s settings.
test_lambda(tag = "parity1", payload = list(number = 2))

### Invoke deployed lambda
deploy_lambda(tag = "parity1", Timeout = 30)

``` r
invoke_lambda(
function_name = "parity1",
invocation_type = "RequestResponse",
payload = list(number = 2),
include_logs = FALSE
payload = list(number = 2)
)

#> Lambda response payload:
#> {"parity":"even"}
```

### Run on a schedule

`schedule_lambda()` runs a deployed function at a `rate()` or `cron()`
[schedule
expression](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-create-rule-schedule.html):

``` r
schedule_lambda("parity1", "rate(5 minutes)")
```

A scheduled run doesn’t take a payload. `{lambdr}` passes the `detail`
of the EventBridge event to the handler as arguments, and for a schedule
that is empty, so the handler is called with no arguments. `parity()`
needs `number`, so each scheduled run of `parity1` fails. Give the
arguments of a scheduled handler defaults, e.g.
`parity <- function(number = 2)`.

### Clean up

`delete_lambda()` removes what the steps above created: the function
and its schedules, the execution role, the ECR repository with its
images, the function’s log group, and the local Docker image.

``` r
delete_lambda("parity1")

# Keep the images, for example to deploy them again later
delete_lambda("parity1", keep_ecr_image = TRUE, keep_local_image = TRUE)
```

A role you passed with `deploy_lambda(role_arn = )` is never deleted.
[Get started](https://discindo.github.io/r2lambda/articles/r2lambda.html)
explains each step and its options, including IAM policies, schedules
and multi-file projects. The
[articles](https://discindo.github.io/r2lambda/articles/) show complete
examples.

## Code of Conduct

Expand Down
3 changes: 3 additions & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ development:
mode: release

articles:
- title: Get started
contents:
- articles/r2lambda
- title: Tutorials
contents:
- articles/tt-lambda
Expand Down
11 changes: 6 additions & 5 deletions cran-comments.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@

## Examples

The `aws_connect()` example runs. The other examples are wrapped in
`\dontrun{}`: they need AWS credentials, and they create, invoke or delete
billable AWS resources (ECR repositories, IAM roles, Lambda functions and
EventBridge schedules). `build_lambda()`, `test_lambda()`, `deploy_lambda()`
and `delete_lambda()` also need a running Docker daemon.
The `aws_connect()` example runs: it only creates a 'paws' client object,
which needs no credentials and sends no request to AWS. The other examples
are wrapped in `\dontrun{}`: they need AWS credentials, and they create,
invoke or delete billable AWS resources (ECR repositories, IAM roles, Lambda
functions and EventBridge schedules). `build_lambda()`, `test_lambda()`,
`deploy_lambda()` and `delete_lambda()` also need a running Docker daemon.
Loading
Loading