Moving your workloads to the cloud can unlock powerful potential, but security can't take a back seat during the transition. For businesses undergoing an AWS cloud transformation, seamlessly integrating Security Information and Even Monitoring (SIEM) with your new cloud environment is crucial. This holistic approach starts with a robust Security Operations Center (SOC) and layered security best practices.

Integrating AWS with on-prem SIEM platform

Customers who like to leverage their existing on-prem (in-house) SIEM tool for real-time log analytics and security event monitoring need to integrate their SIEM tool with AWS. Here are some integration patterns.
S3 bucket directory prefix patternIn this pattern we configure the centralised Amazon S3 directory prefix in the on-prem SIEM tool. All the AWS cloud resources such as Amazon GuardDuty, AWS CloudTrail, Amazon EC2 sends the logs to the centralised S3 bucket. The on-prem SIEM tool checks the configured S3 directory on periodic basis. When new log files are added to the S3 bucket, the SIEM tool pulls those log files for further processing. The frequency for polling the S3 directories impacts the freshness of the logs and the performance of the SIEM.

We have depicted this pattern in Figure 4.

Tech blog Amazon India
Figure 4 SIEM Integration with Amazon S3 using directory prefix

Event notification pattern

In this pattern, whenever a new object is added/updated to the centralised Amazon S3 bucket, an event is generated that notifies the configured Amazon SQS queue. The on-prem SIEM is configured with the SQS queue and gets the added/updated object from S3 using event. The pattern avoids polling and the SIEM gets the object once it is created or updated.

We have depicted this pattern in Figure 5.

Amazon India tech blog
Figure 5 SIEM Integration using SQS

Integration with AWS Security Hub PatternThe on-prem SIEM platform can be integrated with AWS Security Hub and it send and receive the notifications as depicted in Figure 6.

Amazon India tech blog
Figure 6 Integrating on-prem SIEM with AWS Security Hub

AWS native services such as Amazon GuardDuty, Amazon Macie, Amazon Inspector sends the security events to the AWS Security Hub. Amazon GuardDuty sends the threats and anomalous behaviour details; Amazon Macie sends the identified sensitive data details and Amazon inspector sends the identified vulnerabilities to AWS Security Hub.

Amazon Detective can be used to investigate the events and findings from the AWS Security Hub. The on-prem SIEM platforms can be natively integrated with AWS Security Hub to receive and send notifications.

For instance, IBM QRadar provides a bidirectional integration with AWS Security Hub by sending QRadar offenses to and consuming findings from Security Hub for investigation and remediation that provides the following benefits -

  • Consolidate data collection of security data from AWS
  • Deliver real-time multi-environment detection, correlation, and threat intelligence
  • Augment cloud-native security context with IBM QRadar’s advanced security analytics

Event collectors and forwardersIn this pattern we install a native SIEM event collector in AWS. The Event collector collects all the events of interest from AWS and forwards the events to the on-prem SIEM platform.

Subscribing to marketplace apps

AWS provides many popular third party SIEM products such as Securonix, Logrhythm in the marketplace. Customers can subscribe to the marketplace apps to quickly deploy the cloud-native SIEM products to the AWS cloud.

Cloud Security Posture Management (CSPM) on AWS

CSPM involves tools and services that do continuous monitoring and identification of the misconfigurations, compliance violations and other risks on the cloud. CSPM services report, alert the findings and have mechanisms to auto-remediate the identified violations.

Organizations use CSPM service to ensure timely security incident identification, reporting and to ensure regulatory compliance.

AWS Security Hub as CSPM

AWS Security Hub is the CSPM that monitors the AWS services and configurations and reports the security issues in a portal. Given below are the key CSPM features enabled by AWS security Hub –

  • AWS Security Hub has a defined set of security checks such as AWS Foundational security best practices, compliance checks for industry standards such as PCI-DSS and CIS.
  • AWS security hub continuously monitors the AWS resources in real time against the configured security checks.
  • AWS Security Hub aggregates the security configuration issues from various services (such as Amazon Macie, Amazon Inspector, AWS Config and others).
  • AWS Security Hub also can remediate the security issues.
  • AWS Security Hub alerts the identified security issues and reports them in the security hub portal.

We have depicted the role of AWS Security Hub as CSPM in Figure 7. AWS config feed the configuration-related security checks on Amazon EC2 to AWS Security Hub. Amazon Macie, Amazon Inspector feed the findings to AWS Security Hub. Amazon GuardDuty feeds the threat intelligence findings from VPC flow logs, DNS logs, AWS CloudTrail logs and third-party feeds to AWS SecurityHub. Additionally, we can directly feed the logs and alerts from 30+ partner products into AWS SecurityHub. The consolidated and prioritized findings from AWS SecurityHub can be fed into AWS Security Lake for further analysis.

Amazon India Tech blog
Figure 7 AWS Security Hub as CSPM

Best practices during build and integration of SIEM on AWS

In this section we have detailed the best practices during build and integration of SIEM on AWS –

Transfer findings instead of raw information

When we integrate with on-prem SIEM products, instead of sending the raw data, we can only send the findings and insights to the on-prem SIEM platform. This method reduces the data transfer out cost and improves the performance of the SIEM.

Leverage cloud-native services for insights

AWS native services such as Amazon Inspector and Amazon GuardDuty can efficiently detect the patterns based on the metrics and traffic data. AWS native services also provide the contextual insights. Hence it is recommended to leverage the AWS native services to quickly analyse and generate actionable insights for cloud services.


We recommend to automate the end to end processes related to SIEM. Logging, monitoring, data transfer to SIEM platform, alert generation, notification should be automated.

Avoid duplicate analysis

Cloud native tools such as Amazon GuardDuty and Amazon Inspector generate the contextual insights for AWS Services. Instead of generating the insights both at AWS end and at SIEM platform end, we can avoid duplicacy of the insights. We can leverage cloud native tools for generating the insights and actionable alerts for AWS resources.

GenAI in security operations and control

Generative AI (GenAI) is enabling enterprises in automation and content generation. We can leverage GenAI to translate natural language queries to SQL queries and fetch the structured data from the underlying databases. As a result, GenAI democratizes the database access.

We can leverage the natural language to SQL translation feature to improve the security posture. The level 1 (L1) security support team can use Amazon Titan, a GenAI model to query the security events from the security lake.

Appendix – 1 Key metrics to be monitored

Application Load Balancers (ALB)

Below are some of CloudWatch metrics:

  • Target response time
  • Number of requests
  • Target connections errors
  • Active connection counts
  • Processed bytes
  • Sum of rejected connections

Auto-Scaled EC2 Instances

Below are some of CloudWatch metrics:

  • CPU utilization
  • Disk reads and writes (bytes)
  • Disk read operations (operations)
  • Network in and out (Bytes)
  • Status check failed (instances)
  • For auto-scaling groups:
    • Standby instances
    • Terminating instances
    • Minimum and maximum group size
    • Desired capacity
  • Total capacity unit

RDS SQL Server

Below are some of CloudWatch metrics for RDS SQL Server:

  • Performance baseline
    • Network throughput
    • Client connections
    • I/O for Read, Write or Metadata operations
    • Burst credit balances for your DB instances
  • Performance guidelines
    • High CPU or RAM consumption
    • Disk space consumption
    • Network traffic
    • Database connections
    • IOPS metrics

Appendix – 2: Event monitoring for thresholds

ResourceSecurity alertAlert name and trigger conditionNotes
ALB instanceNoRejectedConnectionCountCloudWatch alarm if the number of connections that were rejected because the load balancer reached its maximum.
sum > 0 for 1 min, 5 consecutive times.
ALB targetNoTargetConnectionErrorCountCloudWatch alarm if number of connections were unsuccessfully established between the load balancer and the registered instances.
sum > 0 for 1 min, 5 consecutive times.
HTTPCode_Target_5XX_CountCloudWatch alarm on excess number of HTTP 5XX response codes generated by the targets.
sum > 0 for 1 min, 5 consecutive times.
EC2 instance - all OSesNoCPUUtilizationCloudWatch alarm. High CPU utilization is an indicator of a change in application state such as dead locks, infinite loops, malicious attacks, and other anomalies.
>= 95% for 5 mins, 6 consecutive times.
StatusCheckFailedCloudWatch alarm.
> 0 for 5-minute , 3 consecutive times.
Root Volume Usage
>= 95% for 5 mins, 6 consecutive times.
Memory Free
MemoryFree < 5% for 5 minutes, 6 consecutive times.
YesEPS MalwareCloudWatch event.
Malware found on instance.
EC2 instance - LinuxNoRoot Volume Inode UsageCloudWatch alarm. Applied to Linux instances only.
Average >= 95% for 5 mins, 6 consecutive times.
Swap Free
Memory Swap < 5% for 5 minutes , 6 consecutive times.
Managed Active DirectoryNoActive Directory StatusService event. Emitted when the directory is operating normally after an event.
Managed AD instance sends an active status event.
Impaired Directory StatusService event. Emitted when the directory is running in a degraded state. One or more issues have been detected, and not all directory operations may be working at full operational capacity.
Managed AD instance sends an impaired directory status event.
Inoperable Directory StatusService event. Emitted when the directory is not functional. All directory endpoints have reported issues.
Managed AD instance sends an inoperable status event.
Deleting Directory StatusService event. Emitted when the directory is currently being deleted.
Managed AD instance sends a deleting directory status event.
Failed Directory StatusService event. Emitted when the directory could not be created.
Managed AD instance sends a failed status event.
RestoreFailed Directory StatusService event. Emitted when restoring the directory from a snapshot failed.
Managed AD instance sends a restore failed directory status event.
RDS instanceNo
CPUUtilizationCW alarm.
Average CPU utilization > 75% for 15 mins, 2 consecutive times.
Sum is > 75 for 1 mins, 2 consecutive times.
Average < 1,073,741,824 bytes for 5 mins, 2 consecutive times.
Average >= 1.001 seconds for 5 mins, 2 consecutive times.
Average >= 1.005 seconds for 5 mins, 2 consecutive times.
Average >= 104,857,600 bytes for 5 mins, 2 consecutive times.

Appendix – 3 Key events to be monitored

Enterprises sometimes use Hybrid cloud environment wherein some of the systems are deployed on-prem and few applications on the Cloud. In the hybrid cloud environment, we need to monitor the security events across on-prem and cloud holistically. In this section we detail the main events that are monitored through the SIEM solution during the hybrid cloud scenario.

Core Events for SIEM platform

We have given the core events collected and monitored during the hybrid cloud scenario –

<b>Event to be monitored</b><b>Brief Details</b><b>How to monitor the event on AWS</b>
Data movement from Cloud Storage ObjectWe monitor the data sharing, data exfiltration attempts.Enable AWS CloudTrail event logging for Amazon S3 objects.
Abuse Elevation Control MechanismWe monitor the horizontal and vertical escalation of privilegesWe use AWS IAM access policies, roles and permission boundaries to enforce access restriction. We use temporary tokens using AWS STS instead of long-lived tokens
Account Access RemovalWe monitor the account related events such as deletion, disable and such.AWS account events are monitored through CloudTrail. We enable Multi-factor authentication (MFA) for AWS administrators who have account management privileges
Account ManipulationWe monitor the credential or access change events for the accountWe enable MFA for sensitive operations such as password change, privilege change etc. We also audit the account change events.
Brute Force attemptsWe monitor the brute force login attemptsWe enable password policies and restrict the login attempts and enable MFA for untrusted sources. We also use AWS WAF and AWS GuardDuty, CAPTCHA to detect and prevent bruteforce attacks.
Cloud Infrastructure DiscoveryWe monitor the operations and APIs that list the servers, accounts and other cloud resourcesWe restrict the access to the sensitive operations through AWS IAM policy and generate automated alert during suspicious usage of such operations.
cloud Service DashboardWe monitor the AWS cloud login eventsWe restrict the access to the AWS dashboard and enable MFA for such operations.
Data DestructionWe monitor the data deletion eventsWe enable various security features such as regular backups, database snapshots, S3 MFA delete, S3 object versioning and
Exfiltration over C2 ChannelWe monitor the data exfiltration attemptsWe enable VPC flowlogs and Amazon GuardDuty to monitor the data exfiltration attempts and configure automated remediation measures using Lambda
Exploitation for Privilege EscalationWe monitor the horizontal and vertical escalation of privilegesWe use AWS IAM access policies, roles and permission boundaries to enforce access restriction. We use temporary tokens using AWS STS instead of long-lived tokens
Impair defences attemptsWe monitor the changes to the infrastructure configuration such as firewall rule change, port change and such.We enforce the preventative guardrails at the account level and restrict the changes to Firewall rules, CloudTrail rules, CloudWatch logging rules and others.
Network Sniffing attemptsWe monitor the network sniffing attemptsWe enable network security controls such as Network firewall, VPC Flow logs, VPC traffic mirroring, Amazon GuardDuty and such
Non-Standard Ports usageWe monitor the port changes to the serversWe configure the preventative and detective guardrails at the account level to restrict and monitor the ports
Proxy anomalies
Resource HijackingWe monitor the AWS resource usage for anomalies and known patterns such as crypto miningWe use Amazon GuardDuty to monitor the suspicious activity and alert
Subvert Trust ControlsWe use only the trusted resourcesWe use methods such as code signing using AWS Signer, certificate-based authentication.
Valid/Invalid AccountsWe monitor the account and role usage.We use principle of least privilege and use tools such as IAM access analyser to monitor the AWS account usage and take corrective actions.

We acknowledge and sincerely thank Mr. Ratan Jyoti, CISO, Ujjivan SFB for his inputs to "Drivers for Security Operation and controls (SOC) and SIEM" and "Core Events for SIEM platform" sections of the blog.

DISCLAIMER: The information in this article about software testing techniques and best practices is provided for general educational purposes only. The author is not an attorney and this article does not constitute legal advice. Readers should consult an attorney if they need legal advice about software testing or any other topic covered in this article. The author and publisher disclaim any liability arising from reliance on the information contained herein. The software/ tools and brand names mentioned in this article are trademarks or registered trademarks of their respective owner entities. The author is not affiliated with or sponsored by any of the third party software owners mentioned. The reference to third party software is for informational purposes only and does not constitute an endorsement or recommendation.