From afbc96fe2963df405d27ec8d53c3443ae8a05075 Mon Sep 17 00:00:00 2001 From: novica Date: Sat, 7 Nov 2026 17:56:13 +0100 Subject: [PATCH] chore: prepare README and NEWS for the first CRAN release --- NEWS.md | 99 ++++----------------- README.md | 148 ++++++++------------------------ _pkgdown.yml | 3 + cran-comments.md | 11 +-- vignettes/articles/r2lambda.qmd | 141 ++++++++++++++++++++++++++++++ 5 files changed, 199 insertions(+), 203 deletions(-) create mode 100644 vignettes/articles/r2lambda.qmd diff --git a/NEWS.md b/NEWS.md index 31e41f3..e40ab19 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,92 +1,23 @@ # r2lambda 0.3.0 -## New features +* First CRAN release. -* raise errors with cli ([#49](https://github.com/discindo/r2lambda/issues/49)) ([b72e349](https://github.com/discindo/r2lambda/commit/b72e349f9fdb86413910bd8dda842cf41ce83875)) +* `build_lambda()` builds a Docker image for an R runtime script, with R + packages from a list or an `renv.lock` file, Amazon Linux 2023 system + packages, support files and folders, and `x86_64` or `arm64` + architecture. +* `test_lambda()` runs the image in a local container and invokes it. -## Bug fixes +* `deploy_lambda()` pushes the image to the AWS Elastic Container Registry + and creates the Lambda function, with a new execution role or an + existing one. -* check deploy_lambda() inputs before pushing the image ([#59](https://github.com/discindo/r2lambda/issues/59)) ([691a9f8](https://github.com/discindo/r2lambda/commit/691a9f80e967a1bde7b083c07608303d534b6bd4)) -* keep schedule rule and statement names distinct and within limits ([#56](https://github.com/discindo/r2lambda/issues/56)) ([13c8dc3](https://github.com/discindo/r2lambda/commit/13c8dc3f6a239a4d8d3010e5aedb400c25141b8d)) -* limit delete_lambda() to the current region's unused roles ([#57](https://github.com/discindo/r2lambda/issues/57)) ([aa17fe9](https://github.com/discindo/r2lambda/commit/aa17fe93411615433c53688cfaaea7b0583d4919)) -* restore renv lockfiles from their own repositories ([#58](https://github.com/discindo/r2lambda/issues/58)) ([2d9b944](https://github.com/discindo/r2lambda/commit/2d9b944477e9616b460bb024ea160d4f5cd43430)) -* retry test_lambda() requests on any curl error ([#55](https://github.com/discindo/r2lambda/issues/55)) ([2cc00f5](https://github.com/discindo/r2lambda/commit/2cc00f542281c8b6611543f821f39250e33469c8)) +* `invoke_lambda()` invokes a deployed function, `schedule_lambda()` runs + it on an EventBridge schedule, and `delete_lambda()` removes its AWS + resources and local image. -# r2lambda 0.2.0 - -## New features - -* add delete_lambda() to remove a function's AWS resources ([#41](https://github.com/discindo/r2lambda/issues/41)) ([13abc85](https://github.com/discindo/r2lambda/commit/13abc855b020902f7f14a916fcd34ce72f7d40d8)) - - -## Bug fixes - -* keep IAM role names within 64 characters ([#46](https://github.com/discindo/r2lambda/issues/46)) ([f21a0bd](https://github.com/discindo/r2lambda/commit/f21a0bde854faaaf0ed48af2c8dd41cc044a7e18)) -* send EventBridge target ID as a string ([#40](https://github.com/discindo/r2lambda/issues/40)) ([d69201a](https://github.com/discindo/r2lambda/commit/d69201a6c0f43d3e609556c16198e83e84e9cd81)) - - -## Documentation - -* convert vignettes to Quarto ([#37](https://github.com/discindo/r2lambda/issues/37)) ([2fd68c7](https://github.com/discindo/r2lambda/commit/2fd68c7d1e9506929859dd1596c6b0d48063952c)) -* group articles under a Tutorials heading ([#45](https://github.com/discindo/r2lambda/issues/45)) ([624fbbd](https://github.com/discindo/r2lambda/commit/624fbbd60ec7c0893f2960c33b3ab4a7b34f224b)) -* maintain README.md directly and drop README.Rmd ([#39](https://github.com/discindo/r2lambda/issues/39)) ([3b2ff98](https://github.com/discindo/r2lambda/commit/3b2ff98a52bc36b4f195308ea18a053a29876ccf)) -* meet CRAN extra checks ([#42](https://github.com/discindo/r2lambda/issues/42)) ([4968dbf](https://github.com/discindo/r2lambda/commit/4968dbf65a62545b6cf2ff92a87207ac62df9630)) -* move vignettes to pkgdown-only articles ([#43](https://github.com/discindo/r2lambda/issues/43)) ([c958ce6](https://github.com/discindo/r2lambda/commit/c958ce6538438f136b1961fba66bb21709667929)) - -# r2lambda 0.1.0 - -## New features - -* accept folders in support_path ([27254c3](https://github.com/discindo/r2lambda/commit/33725e644636c1a8be2c10793192dce5b6e53308)) -* add renv logic when creating the dockerfile ([5e94146](https://github.com/discindo/r2lambda/commit/a7d42fff132825c7c4af16eebbcc05ab705969c2)) -* allow support files (multi-file project) ([8180779](https://github.com/discindo/r2lambda/commit/5fd71e166887344d66ec9e5d6c19c38b8b49058a)) -* allow support files (multi-file project) ([95f9949](https://github.com/discindo/r2lambda/commit/44379b375e7961912ead279293ebde343549df06)) -* build and deploy arm64 Lambda functions ([d9b3bde](https://github.com/discindo/r2lambda/commit/c185b116a317dad6fbaa25e67e2aef642fb51c2b)) -* check lambda state before invoking ([bf37034](https://github.com/discindo/r2lambda/commit/bf37034d91204f71a33887ba13d2d7c637d700f5)) -* deploy with an existing IAM role ([8cfe969](https://github.com/discindo/r2lambda/commit/fc25d4d0b35e6969d87f97d5689caddfc729d0c1)), closes [#6](https://github.com/discindo/r2lambda/issues/6) -* grant AWS access through the Lambda execution role ([1ec810a](https://github.com/discindo/r2lambda/commit/0537753e829102e4e2b2b4581dbbc5855a608f6d)) -* install system packages in the Lambda image ([3d642a6](https://github.com/discindo/r2lambda/commit/256e40f01938cda26531868436b4ebb46bfbe8b3)), closes [#5](https://github.com/discindo/r2lambda/issues/5) -* parse ecr token. ([9adf88a](https://github.com/discindo/r2lambda/commit/9adf88a18f1603e0378b6c674f345fe2e245f4bc)) -* separate build from deploy ([e2847ca](https://github.com/discindo/r2lambda/commit/e2847cacf48b0a43652ae55cb7cb3ddd6aa7bc47)) -* start, test, and stop a lambda container locally ([d252680](https://github.com/discindo/r2lambda/commit/97862a6718e409b985f6e1e01efb9124ffe4986b)) -* start, test, and stop a lambda container locally ([dc213d2](https://github.com/discindo/r2lambda/commit/a499635b9a698009d4b54bbd3dfe22b7ed6f5333)) -* try to catch failed invocation, including due to Inactive state ([881f329](https://github.com/discindo/r2lambda/commit/881f329d98b96ffb06a0304b98386554dfa99948)) -* update lambda to run on schedule ([7f4472c](https://github.com/discindo/r2lambda/commit/456a6ab7ba173b179b01bfad0fcc8abffeb8b351)) -* use magrittr pipe ([6fa8575](https://github.com/discindo/r2lambda/commit/6fa85759d5d60ee9f1d192e497eb8b1c75d81282)) -* use stevedore for push to ecr ([3e6641a](https://github.com/discindo/r2lambda/commit/3e6641a1cdf5db235a1d35969833e77582d2eab3)) -* use stevedore to create the docker image ([a12a491](https://github.com/discindo/r2lambda/commit/a12a491e987cb256e5e257f704f8dc096af9e83c)) - - -## Bug fixes - -* build Lambda images on Amazon Linux 2023 with R 4.6.1 ([d573317](https://github.com/discindo/r2lambda/commit/161a5f9fc76a09462490d1461b4f5ef57faf261e)) -* call the docker CLI instead of stevedore ([c711055](https://github.com/discindo/r2lambda/commit/35b96e65825f84c768eae040a89a90500e52f63f)), closes [#11](https://github.com/discindo/r2lambda/issues/11) -* check the image architecture before pushing it ([d74aaf3](https://github.com/discindo/r2lambda/commit/03f7703647fc60238eeead7c6ea8bfbc80b89e7b)) -* correct log msg ([79a3df0](https://github.com/discindo/r2lambda/commit/79a3df0cce63ea9f535102c8d701eceb7a6eac49)) -* don't create an ECR repository when deploying ([a669cef](https://github.com/discindo/r2lambda/commit/6d20439192488c1fb28f3232f1ddc5647e630069)) -* don't need to check for aws cli anymore ([673b69c](https://github.com/discindo/r2lambda/commit/673b69c7720182b3c6bf87d16a5c963d2fd2f741)) -* inspect the same image that deploy_lambda pushes ([ec91125](https://github.com/discindo/r2lambda/commit/26b635b9afd6c9900054b35a24a3a25d402c4e73)) -* install from github in readme ([61e6435](https://github.com/discindo/r2lambda/commit/61e643567f470e9d556cc476c00ddddcfe900565)) -* look up AWS resources by name instead of listing them ([a99bfc0](https://github.com/discindo/r2lambda/commit/5b7f38d4480e9ce7397365c59862cf655821228d)) -* name support files in the Dockerfile as they are copied ([bbf5296](https://github.com/discindo/r2lambda/commit/0766ac1f3d1f63edf80c1edc7e787d935c6fe8dc)) -* report AWS errors from invoke_lambda and deploy_lambda ([1ceef40](https://github.com/discindo/r2lambda/commit/28912ffe69b2da60d51723ede8e989b8c23883f7)) -* tempdir issue in tests ([28032d9](https://github.com/discindo/r2lambda/commit/ac5a18d58c113fcf01feee7469d92251a8b17a00)) - - -## Documentation - -* add Novica Nakov as maintainer and ORCIDs for both authors ([9c20ff7](https://github.com/discindo/r2lambda/commit/6b9b8a3c25ef238b5ff089e7b97d0ad707574aed)) -* build ignore docs folder ([8a5af02](https://github.com/discindo/r2lambda/commit/8a5af02af3cd512f79815d8c15e1a64d8992bd7e)) -* code of conduct. ([9608875](https://github.com/discindo/r2lambda/commit/96088754bc19dd9a7df75494b2cd4b730d0f4ed8)) -* correction in readme ([55b077a](https://github.com/discindo/r2lambda/commit/55b077a32360e4fd2bc57df16edf61619492f80b)) -* document role, policy, support folder, system package and architecture options ([e2a5d07](https://github.com/discindo/r2lambda/commit/058096088f9ee4282883852c08c70962ef901490)) -* fix moved blog link in lambda-s3 vignette ([4471463](https://github.com/discindo/r2lambda/commit/8c06a4005b8c891b85ae044127470eea81686437)) -* manual files ([16ba1e2](https://github.com/discindo/r2lambda/commit/16ba1e261a38186613853031b5096d90864be44d)) -* manual files ([4c16601](https://github.com/discindo/r2lambda/commit/4c16601c2b3a4c23447efa52d148401f577d682b)) -* manual files and vignette ([f470ef7](https://github.com/discindo/r2lambda/commit/815449e7a6e18a47b8d928e63675af99d26fab4b)) -* pkg description ([9b778d1](https://github.com/discindo/r2lambda/commit/9b778d1f880d4ccf29ac6823b63de0c3ebdb6625)) -* typo ([41386a6](https://github.com/discindo/r2lambda/commit/41386a64fb6c76979dff7df16fdde333fd420882)) -* update readme ([a002958](https://github.com/discindo/r2lambda/commit/71927624c861f1f145aa3445a0b95055b26d4d9d)) -* update readme ([f89c595](https://github.com/discindo/r2lambda/commit/f89c5957604c8a2ceec9334d959ca691d150add1)) +* `aws_connect()` creates a `paws` client for an AWS service. +Changes made before the first CRAN release are listed in +[CHANGELOG.md](https://github.com/discindo/r2lambda/blob/main/CHANGELOG.md). diff --git a/README.md b/README.md index b3a0e09..95fd11a 100644 --- a/README.md +++ b/README.md @@ -10,18 +10,23 @@ coverage](https://codecov.io/gh/discindo/r2lambda/graph/badge.svg)](https://app. The goal of `{r2lambda}` is to make it easier to go from an `R` script to a deployed `AWS Lambda` function. -## Requirements +## System requirements -- [docker](https://docs.docker.com/get-docker/) with the +`{r2lambda}` calls two programs outside R. They are the same on Linux, +macOS and Windows; only how you install them differs. + +- [Docker](https://docs.docker.com/get-docker/) with the [buildx](https://docs.docker.com/build/concepts/overview/#buildx) - plugin is required to build, tag, and push the image. Docker Desktop - includes buildx; on Linux, some distributions package it separately, - e.g. `docker-buildx`. -- [curl](https://curl.se/) 7.71 or later is required by `test_lambda()` - to call the local container. -- An AWS account. `build_lambda()` already creates the ECR repository - for the image, and `deploy_lambda()` creates an IAM role and the - Lambda function. + plugin, to build, run and push images. On macOS and Windows, install + Docker Desktop, which includes buildx; on Windows, use its default + Linux containers. On Linux, install Docker Engine and the buildx + plugin, which some distributions package separately, e.g. + `docker-buildx`. The `docker` command must be on the `PATH` and the + Docker daemon must be running. +- [curl](https://curl.se/) 7.71 or later, used by `test_lambda()` to + call the local container. macOS and current Windows 10 and 11 include + it; on Linux, install it with your package manager. Check the version + with `curl --version`. ## Installation @@ -40,6 +45,10 @@ remotes::install_github("discindo/r2lambda") ## Setup +You need an AWS account. `build_lambda()` creates the ECR repository +for the image, and `deploy_lambda()` creates an IAM role and the Lambda +function. + `r2lambda` assumes credentials for connecting to AWS services are available in the `R` session. This can be done via an `.Renviron` file that should set environmental variables like so: @@ -54,130 +63,41 @@ methods supported by `paws` are available in `r2lambda`. See [here](https://github.com/paws-r/paws/blob/main/docs/credentials.md) for details on setting credentials, region, profile, etc. -## Workflow - -### Build a docker image for the lambda function +## Example -The examples use two files installed with the package: -`parity.R`, a runtime script whose `parity()` function tells whether a -number is even or odd, and `renv.lock`, an example lockfile. -`system.file()` returns their paths. Replace them with your own script -and lockfile. +Build an image from `parity.R`, an example runtime script installed +with the package, test it locally, deploy it, invoke it, and delete it: ``` r -runtime_function <- "parity" -runtime_path <- system.file("parity.R", package = "r2lambda") -renvlock_path <- system.file("renv.lock", package = "r2lambda") -dependencies <- NULL +library(r2lambda) -# Might take a while, its building a docker image build_lambda( tag = "parity1", - runtime_function = runtime_function, - runtime_path = runtime_path, - renvlock_path = renvlock_path, - dependencies = dependencies -) -``` - -`build_lambda()` has a few more options: - -- `support_path`: files or folders the runtime script needs. Each is - copied next to the runtime script under its own name, so - `support_path = "R"` lets the script call `source("R/helpers.R")`. -- `system_deps`: Amazon Linux 2023 packages to install with `dnf`, for R - packages that need system libraries, e.g. `system_deps = "udunits2"` - for `units`. Amazon Linux 2023 has no EPEL, so some libraries, such as - GDAL, are not available. -- `architecture`: `"x86_64"` (default) or `"arm64"`. `deploy_lambda()` - creates the function with the image’s architecture. Building for an - architecture other than your machine’s needs Docker’s QEMU emulation. - -### Test the lambda docker image locally - -``` r -payload <- list(number = 2) -tag <- "parity1" -test_lambda(tag = "parity1", payload) -``` - -### Deploy to AWS Lambda - -``` r -# Might take a while, its pushing it to a remote repository -deploy_lambda(tag = "parity1") -``` - -Lambda stops a function after 3 seconds by default, which an R -container may not meet, especially on its first call. Pass a longer -`Timeout`, in seconds, and other `create_function()` settings through -`...`: - -``` r -deploy_lambda(tag = "parity1", Timeout = 30, MemorySize = 512) -``` - -The function runs under a new IAM role with basic execution permissions. -To let it call other AWS services, attach policies to that role, or use -a role you already have: - -``` r -deploy_lambda( - tag = "parity1", - policy_arns = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess" + runtime_function = "parity", + runtime_path = system.file("parity.R", package = "r2lambda"), + renvlock_path = system.file("renv.lock", package = "r2lambda") ) -deploy_lambda(tag = "parity1", role_arn = "arn:aws:iam::123456789012:role/my-role") -``` - -Inside the function, `paws` picks up the role’s temporary credentials, -so no AWS keys go into the image or the function’s settings. +test_lambda(tag = "parity1", payload = list(number = 2)) -### Invoke deployed lambda +deploy_lambda(tag = "parity1", Timeout = 30) -``` r invoke_lambda( function_name = "parity1", invocation_type = "RequestResponse", - payload = list(number = 2), - include_logs = FALSE + payload = list(number = 2) ) - #> Lambda response payload: #> {"parity":"even"} -``` - -### Run on a schedule -`schedule_lambda()` runs a deployed function at a `rate()` or `cron()` -[schedule -expression](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-create-rule-schedule.html): - -``` r -schedule_lambda("parity1", "rate(5 minutes)") -``` - -A scheduled run doesn’t take a payload. `{lambdr}` passes the `detail` -of the EventBridge event to the handler as arguments, and for a schedule -that is empty, so the handler is called with no arguments. `parity()` -needs `number`, so each scheduled run of `parity1` fails. Give the -arguments of a scheduled handler defaults, e.g. -`parity <- function(number = 2)`. - -### Clean up - -`delete_lambda()` removes what the steps above created: the function -and its schedules, the execution role, the ECR repository with its -images, the function’s log group, and the local Docker image. - -``` r delete_lambda("parity1") - -# Keep the images, for example to deploy them again later -delete_lambda("parity1", keep_ecr_image = TRUE, keep_local_image = TRUE) ``` -A role you passed with `deploy_lambda(role_arn = )` is never deleted. +[Get started](https://discindo.github.io/r2lambda/articles/r2lambda.html) +explains each step and its options, including IAM policies, schedules +and multi-file projects. The +[articles](https://discindo.github.io/r2lambda/articles/) show complete +examples. ## Code of Conduct diff --git a/_pkgdown.yml b/_pkgdown.yml index e3152e6..69aa128 100644 --- a/_pkgdown.yml +++ b/_pkgdown.yml @@ -8,6 +8,9 @@ development: mode: release articles: + - title: Get started + contents: + - articles/r2lambda - title: Tutorials contents: - articles/tt-lambda diff --git a/cran-comments.md b/cran-comments.md index 2e12ed3..8fa3361 100644 --- a/cran-comments.md +++ b/cran-comments.md @@ -6,8 +6,9 @@ ## Examples -The `aws_connect()` example runs. The other examples are wrapped in -`\dontrun{}`: they need AWS credentials, and they create, invoke or delete -billable AWS resources (ECR repositories, IAM roles, Lambda functions and -EventBridge schedules). `build_lambda()`, `test_lambda()`, `deploy_lambda()` -and `delete_lambda()` also need a running Docker daemon. +The `aws_connect()` example runs: it only creates a 'paws' client object, +which needs no credentials and sends no request to AWS. The other examples +are wrapped in `\dontrun{}`: they need AWS credentials, and they create, +invoke or delete billable AWS resources (ECR repositories, IAM roles, Lambda +functions and EventBridge schedules). `build_lambda()`, `test_lambda()`, +`deploy_lambda()` and `delete_lambda()` also need a running Docker daemon. diff --git a/vignettes/articles/r2lambda.qmd b/vignettes/articles/r2lambda.qmd new file mode 100644 index 0000000..0ef8522 --- /dev/null +++ b/vignettes/articles/r2lambda.qmd @@ -0,0 +1,141 @@ +--- +title: "Get started with r2lambda" +knitr: + opts_chunk: + collapse: true + comment: "#>" + eval: false +--- + +```{r} +#| label: setup +library(r2lambda) +``` + +This article walks through the whole life of a Lambda function: build a +Docker image, test it locally, deploy it, invoke it, run it on a +schedule and delete it. It assumes Docker, curl and AWS credentials are +set up as described in the +[README](https://discindo.github.io/r2lambda/#system-requirements). + +## Build a docker image for the lambda function + +The examples use two files installed with the package: +`parity.R`, a runtime script whose `parity()` function tells whether a +number is even or odd, and `renv.lock`, an example lockfile. +`system.file()` returns their paths. Replace them with your own script +and lockfile. + +```{r} +runtime_function <- "parity" +runtime_path <- system.file("parity.R", package = "r2lambda") +renvlock_path <- system.file("renv.lock", package = "r2lambda") +dependencies <- NULL + +# Might take a while, its building a docker image +build_lambda( + tag = "parity1", + runtime_function = runtime_function, + runtime_path = runtime_path, + renvlock_path = renvlock_path, + dependencies = dependencies +) +``` + +`build_lambda()` has a few more options: + +- `support_path`: files or folders the runtime script needs. Each is + copied next to the runtime script under its own name, so + `support_path = "R"` lets the script call `source("R/helpers.R")`. +- `system_deps`: Amazon Linux 2023 packages to install with `dnf`, for R + packages that need system libraries, e.g. `system_deps = "udunits2"` + for `units`. Amazon Linux 2023 has no EPEL, so some libraries, such as + GDAL, are not available. +- `architecture`: `"x86_64"` (default) or `"arm64"`. `deploy_lambda()` + creates the function with the image's architecture. Building for an + architecture other than your machine's needs Docker's QEMU emulation. + +## Test the lambda docker image locally + +```{r} +payload <- list(number = 2) +test_lambda(tag = "parity1", payload) +``` + +## Deploy to AWS Lambda + +```{r} +# Might take a while, its pushing it to a remote repository +deploy_lambda(tag = "parity1") +``` + +Lambda stops a function after 3 seconds by default, which an R +container may not meet, especially on its first call. Pass a longer +`Timeout`, in seconds, and other `create_function()` settings through +`...`: + +```{r} +deploy_lambda(tag = "parity1", Timeout = 30, MemorySize = 512) +``` + +The function runs under a new IAM role with basic execution permissions. +To let it call other AWS services, attach policies to that role, or use +a role you already have: + +```{r} +deploy_lambda( + tag = "parity1", + policy_arns = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess" +) + +deploy_lambda(tag = "parity1", role_arn = "arn:aws:iam::123456789012:role/my-role") +``` + +Inside the function, `paws` picks up the role's temporary credentials, +so no AWS keys go into the image or the function's settings. + +## Invoke deployed lambda + +```{r} +invoke_lambda( + function_name = "parity1", + invocation_type = "RequestResponse", + payload = list(number = 2), + include_logs = FALSE +) + +#> Lambda response payload: +#> {"parity":"even"} +``` + +## Run on a schedule + +`schedule_lambda()` runs a deployed function at a `rate()` or `cron()` +[schedule +expression](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-create-rule-schedule.html): + +```{r} +schedule_lambda("parity1", "rate(5 minutes)") +``` + +A scheduled run doesn't take a payload. `{lambdr}` passes the `detail` +of the EventBridge event to the handler as arguments, and for a schedule +that is empty, so the handler is called with no arguments. `parity()` +needs `number`, so each scheduled run of `parity1` fails. Give the +arguments of a scheduled handler defaults, e.g. +`parity <- function(number = 2)`. + +## Clean up + +`delete_lambda()` removes what the steps above created: the function +and its schedules, the execution role, the ECR repository with its +images, the function's log group, and the local Docker image. + +```{r} +delete_lambda("parity1") + +# Keep the images, for example to deploy them again later +delete_lambda("parity1", keep_ecr_image = TRUE, keep_local_image = TRUE) +``` + +A role you passed with `deploy_lambda(role_arn = )` is never deleted.