Deploy AWS Lambda to forward AWS Service logs to vRealize LogInsight Cloud

Recently vRealize LogInsight Cloud announced Multi-Cloud support which allows you to forward logs from AWS and Azure. For more details, you can refer to the blog

post

  

In this blog, I would demonstrate how to deploy AWS Lambda to forward the following AWS service logs to vRealize LogInsight Cloud using

Terraform AWS provider

  • S3 Bucket Logs
  • S3 Bucket Events
  • CloudWatch Logs
Once you execute the Terraform scripts it will create 3 AWS Lambda which will get the logs flowing to vRealize LogInsight Cloud.

What is Terraform

Terraform

 provides Infrastructure as a Code to provision and manage any cloud, infrastructure, or service

Pre-requisites

Following are the pre-requisites
  • Access to VMware vRealize LogInsight Cloud 
  • API Key for VMware vRealize LogInsight Cloud
  • AWS Access Key and ID
    • PowerUser role should be enough for creating the required resources.
  • AWS IAM role which has Lambda Service
    • It needs to exist before executing the script. 
  • S3 Bucket for which you want logs.
    • It needs to exist before executing the script.
  • Cloud Watch Log group for which you want logs.
    • Log group needs to exist before executing the script. 

Download the Lambda Package 

Download Lambda code by executing the below command from a terminal
wget https://github.com/vmware/vmware-log-collectors-for-public-cloud/releases/download/v1.0.3/Lambda.zip
 

Download Terraform Scripts 

Clone following Terraform Repo from my

Github Terraform Repo

 

 

git clone https://github.com/munishpalmakhija/Terraform.git

 

 

 

 
 
 
 
 
Following are the directories for vRLI Cloud 
  • Deploy-vRLICloud-AWS-Lambda-S3BucketLogs
    • It creates AWS Lambda with S3 Trigger when a new object created event happens in the bucket.
  • Deploy-vRLICloud-AWS-Lambda-S3Events
    • It creates AWS Lambda with S3 Trigger when a new object created event happens in the bucket.
  • Deploy-vRLICloud-AWS-Lambda-CloudWatchLogs
    • It creates AWS Lambda with Cloudwatch Trigger (Subscription) 

Deploy AWS Lambda

 

Update the   terraform.tfvars for all 3 scripts

 
Navigate to the folder Deploy-vRLICloud-AWS-Lambda-S3BucketLogs & update the terraform.tfvars with your environment details
Navigate to the folder Deploy-vRLICloud-AWS-Lambda-S3Events & update the terraform.tfvars with your environment details
 
 
Navigate to the folder Deploy-vRLICloud-AWS-Lambda-CloudWatchLogs & update the terraform.tfvars with your environment details
 

Initialize Terraform

 

terraform init
 

 

 
 
 
 
 
 
 
 
 
 
 
 
 
 

Create an Execution Plan 

terraform plan 
 

 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Apply the config to the environment

terraform apply -auto-approve
 

 

 

Once it is successfully executed you will see AWS Lambda Function created as below. 

 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
Similarly, you can execute the other 2 scripts for S3 Bucket Events and Cloudwatch Logs
 

 

 

You should be able to view logs by using the following filter
 
log_type starts with aws
 

 

 
 

0 thoughts on “Deploy AWS Lambda to forward AWS Service logs to vRealize LogInsight Cloud

Leave a Reply

Your email address will not be published.

%d bloggers like this: