Moving your workloads to the cloud can unlock powerful potential, but security can't take a back seat during the transition. For businesses undergoing an AWS cloud transformation, seamlessly integrating Security Information and Even Monitoring (SIEM) with your new cloud environment is crucial. This holistic approach starts with a robust Security Operations Center (SOC) and layered security best practices.
Integrating AWS with on-prem SIEM platform
Customers who like to leverage their existing on-prem (in-house) SIEM tool for real-time log analytics and security event monitoring need to integrate their SIEM tool with AWS. Here are some integration patterns.
S3 bucket directory prefix patternIn this pattern we configure the centralised Amazon S3 directory prefix in the on-prem SIEM tool. All the AWS cloud resources such as Amazon GuardDuty, AWS CloudTrail, Amazon EC2 sends the logs to the centralised S3 bucket. The on-prem SIEM tool checks the configured S3 directory on periodic basis. When new log files are added to the S3 bucket, the SIEM tool pulls those log files for further processing. The frequency for polling the S3 directories impacts the freshness of the logs and the performance of the SIEM.
We have depicted this pattern in Figure 4.
Event notification pattern
In this pattern, whenever a new object is added/updated to the centralised Amazon S3 bucket, an event is generated that notifies the configured Amazon SQS queue. The on-prem SIEM is configured with the SQS queue and gets the added/updated object from S3 using event. The pattern avoids polling and the SIEM gets the object once it is created or updated.
We have depicted this pattern in Figure 5.
Integration with AWS Security Hub PatternThe on-prem SIEM platform can be integrated with AWS Security Hub and it send and receive the notifications as depicted in Figure 6.
AWS native services such as Amazon GuardDuty, Amazon Macie, Amazon Inspector sends the security events to the AWS Security Hub. Amazon GuardDuty sends the threats and anomalous behaviour details; Amazon Macie sends the identified sensitive data details and Amazon inspector sends the identified vulnerabilities to AWS Security Hub.
Amazon Detective can be used to investigate the events and findings from the AWS Security Hub. The on-prem SIEM platforms can be natively integrated with AWS Security Hub to receive and send notifications.
For instance, IBM QRadar provides a bidirectional integration with AWS Security Hub by sending QRadar offenses to and consuming findings from Security Hub for investigation and remediation that provides the following benefits -
- Consolidate data collection of security data from AWS
- Deliver real-time multi-environment detection, correlation, and threat intelligence
- Augment cloud-native security context with IBM QRadar’s advanced security analytics
Event collectors and forwardersIn this pattern we install a native SIEM event collector in AWS. The Event collector collects all the events of interest from AWS and forwards the events to the on-prem SIEM platform.
Subscribing to marketplace apps
AWS provides many popular third party SIEM products such as Securonix, Logrhythm in the marketplace. Customers can subscribe to the marketplace apps to quickly deploy the cloud-native SIEM products to the AWS cloud.
Cloud Security Posture Management (CSPM) on AWS
CSPM involves tools and services that do continuous monitoring and identification of the misconfigurations, compliance violations and other risks on the cloud. CSPM services report, alert the findings and have mechanisms to auto-remediate the identified violations.
Organizations use CSPM service to ensure timely security incident identification, reporting and to ensure regulatory compliance.
AWS Security Hub as CSPM
AWS Security Hub is the CSPM that monitors the AWS services and configurations and reports the security issues in a portal. Given below are the key CSPM features enabled by AWS security Hub –
- AWS Security Hub has a defined set of security checks such as AWS Foundational security best practices, compliance checks for industry standards such as PCI-DSS and CIS.
- AWS security hub continuously monitors the AWS resources in real time against the configured security checks.
- AWS Security Hub aggregates the security configuration issues from various services (such as Amazon Macie, Amazon Inspector, AWS Config and others).
- AWS Security Hub also can remediate the security issues.
- AWS Security Hub alerts the identified security issues and reports them in the security hub portal.
We have depicted the role of AWS Security Hub as CSPM in Figure 7. AWS config feed the configuration-related security checks on Amazon EC2 to AWS Security Hub. Amazon Macie, Amazon Inspector feed the findings to AWS Security Hub. Amazon GuardDuty feeds the threat intelligence findings from VPC flow logs, DNS logs, AWS CloudTrail logs and third-party feeds to AWS SecurityHub. Additionally, we can directly feed the logs and alerts from 30+ partner products into AWS SecurityHub. The consolidated and prioritized findings from AWS SecurityHub can be fed into AWS Security Lake for further analysis.
Best practices during build and integration of SIEM on AWS
In this section we have detailed the best practices during build and integration of SIEM on AWS –
Transfer findings instead of raw information
When we integrate with on-prem SIEM products, instead of sending the raw data, we can only send the findings and insights to the on-prem SIEM platform. This method reduces the data transfer out cost and improves the performance of the SIEM.
Leverage cloud-native services for insights
AWS native services such as Amazon Inspector and Amazon GuardDuty can efficiently detect the patterns based on the metrics and traffic data. AWS native services also provide the contextual insights. Hence it is recommended to leverage the AWS native services to quickly analyse and generate actionable insights for cloud services.
Automation
We recommend to automate the end to end processes related to SIEM. Logging, monitoring, data transfer to SIEM platform, alert generation, notification should be automated.
Avoid duplicate analysis
Cloud native tools such as Amazon GuardDuty and Amazon Inspector generate the contextual insights for AWS Services. Instead of generating the insights both at AWS end and at SIEM platform end, we can avoid duplicacy of the insights. We can leverage cloud native tools for generating the insights and actionable alerts for AWS resources.
GenAI in security operations and control
Generative AI (GenAI) is enabling enterprises in automation and content generation. We can leverage GenAI to translate natural language queries to SQL queries and fetch the structured data from the underlying databases. As a result, GenAI democratizes the database access.
We can leverage the natural language to SQL translation feature to improve the security posture. The level 1 (L1) security support team can use Amazon Titan, a GenAI model to query the security events from the security lake.
Appendix – 1 Key metrics to be monitored
Application Load Balancers (ALB)
Below are some of CloudWatch metrics:
- Target response time
- Number of requests
- Target connections errors
- Active connection counts
- Processed bytes
- Sum of rejected connections
Auto-Scaled EC2 Instances
Below are some of CloudWatch metrics:
- CPU utilization
- Disk reads and writes (bytes)
- Disk read operations (operations)
- Network in and out (Bytes)
- Status check failed (instances)
- For auto-scaling groups:
- Standby instances
- Terminating instances
- Minimum and maximum group size
- Desired capacity
- Total capacity unit
RDS SQL Server
Below are some of CloudWatch metrics for RDS SQL Server:
- Performance baseline
- Network throughput
- Client connections
- I/O for Read, Write or Metadata operations
- Burst credit balances for your DB instances
- Performance guidelines
- High CPU or RAM consumption
- Disk space consumption
- Network traffic
- Database connections
- IOPS metrics
Appendix – 2: Event monitoring for thresholds
| Resource | Security alert | Alert name and trigger condition | Notes |
|---|---|---|---|
| ALB instance | No | RejectedConnectionCount | CloudWatch alarm if the number of connections that were rejected because the load balancer reached its maximum. |
| sum > 0 for 1 min, 5 consecutive times. | |||
| ALB target | No | TargetConnectionErrorCount | CloudWatch alarm if number of connections were unsuccessfully established between the load balancer and the registered instances. |
| sum > 0 for 1 min, 5 consecutive times. | |||
| HTTPCode_Target_5XX_Count | CloudWatch alarm on excess number of HTTP 5XX response codes generated by the targets. | ||
| sum > 0 for 1 min, 5 consecutive times. | |||
| EC2 instance - all OSes | No | CPUUtilization | CloudWatch alarm. High CPU utilization is an indicator of a change in application state such as dead locks, infinite loops, malicious attacks, and other anomalies. |
| >= 95% for 5 mins, 6 consecutive times. | |||
| StatusCheckFailed | CloudWatch alarm. | ||
| > 0 for 5-minute , 3 consecutive times. | |||
| Root Volume Usage | |||
| >= 95% for 5 mins, 6 consecutive times. | |||
| Memory Free | |||
| MemoryFree < 5% for 5 minutes, 6 consecutive times. | |||
| Yes | EPS Malware | CloudWatch event. | |
| Malware found on instance. | |||
| EC2 instance - Linux | No | Root Volume Inode Usage | CloudWatch alarm. Applied to Linux instances only. |
| Average >= 95% for 5 mins, 6 consecutive times. | |||
| Swap Free | |||
| Memory Swap < 5% for 5 minutes , 6 consecutive times. | |||
| Managed Active Directory | No | Active Directory Status | Service event. Emitted when the directory is operating normally after an event. |
| Managed AD instance sends an active status event. | |||
| Impaired Directory Status | Service event. Emitted when the directory is running in a degraded state. One or more issues have been detected, and not all directory operations may be working at full operational capacity. | ||
| Managed AD instance sends an impaired directory status event. | |||
| Inoperable Directory Status | Service event. Emitted when the directory is not functional. All directory endpoints have reported issues. | ||
| Managed AD instance sends an inoperable status event. | |||
| Deleting Directory Status | Service event. Emitted when the directory is currently being deleted. | ||
| Managed AD instance sends a deleting directory status event. | |||
| Failed Directory Status | Service event. Emitted when the directory could not be created. | ||
| Managed AD instance sends a failed status event. | |||
| RestoreFailed Directory Status | Service event. Emitted when restoring the directory from a snapshot failed. | ||
| Managed AD instance sends a restore failed directory status event. | |||
| RDS instance | No | ||
| CPUUtilization | CW alarm. | ||
| Average CPU utilization > 75% for 15 mins, 2 consecutive times. | |||
| DiskQueueDepth | |||
| Sum is > 75 for 1 mins, 2 consecutive times. | |||
| FreeStorageSpace | |||
| Average < 1,073,741,824 bytes for 5 mins, 2 consecutive times. | |||
| ReadLatency | |||
| Average >= 1.001 seconds for 5 mins, 2 consecutive times. | |||
| WriteLatency | |||
| Average >= 1.005 seconds for 5 mins, 2 consecutive times. | |||
| SwapUsage | |||
| Average >= 104,857,600 bytes for 5 mins, 2 consecutive times. |
Appendix – 3 Key events to be monitored
Enterprises sometimes use Hybrid cloud environment wherein some of the systems are deployed on-prem and few applications on the Cloud. In the hybrid cloud environment, we need to monitor the security events across on-prem and cloud holistically. In this section we detail the main events that are monitored through the SIEM solution during the hybrid cloud scenario.
Core Events for SIEM platform
We have given the core events collected and monitored during the hybrid cloud scenario –
| <b>Event to be monitored</b> | <b>Brief Details</b> | <b>How to monitor the event on AWS</b> |
|---|---|---|
| Data movement from Cloud Storage Object | We monitor the data sharing, data exfiltration attempts. | Enable AWS CloudTrail event logging for Amazon S3 objects. |
| Abuse Elevation Control Mechanism | We monitor the horizontal and vertical escalation of privileges | We use AWS IAM access policies, roles and permission boundaries to enforce access restriction. We use temporary tokens using AWS STS instead of long-lived tokens |
| Account Access Removal | We monitor the account related events such as deletion, disable and such. | AWS account events are monitored through CloudTrail. We enable Multi-factor authentication (MFA) for AWS administrators who have account management privileges |
| Account Manipulation | We monitor the credential or access change events for the account | We enable MFA for sensitive operations such as password change, privilege change etc. We also audit the account change events. |
| Brute Force attempts | We monitor the brute force login attempts | We enable password policies and restrict the login attempts and enable MFA for untrusted sources. We also use AWS WAF and AWS GuardDuty, CAPTCHA to detect and prevent bruteforce attacks. |
| Cloud Infrastructure Discovery | We monitor the operations and APIs that list the servers, accounts and other cloud resources | We restrict the access to the sensitive operations through AWS IAM policy and generate automated alert during suspicious usage of such operations. |
| cloud Service Dashboard | We monitor the AWS cloud login events | We restrict the access to the AWS dashboard and enable MFA for such operations. |
| Data Destruction | We monitor the data deletion events | We enable various security features such as regular backups, database snapshots, S3 MFA delete, S3 object versioning and |
| Exfiltration over C2 Channel | We monitor the data exfiltration attempts | We enable VPC flowlogs and Amazon GuardDuty to monitor the data exfiltration attempts and configure automated remediation measures using Lambda |
| Exploitation for Privilege Escalation | We monitor the horizontal and vertical escalation of privileges | We use AWS IAM access policies, roles and permission boundaries to enforce access restriction. We use temporary tokens using AWS STS instead of long-lived tokens |
| Impair defences attempts | We monitor the changes to the infrastructure configuration such as firewall rule change, port change and such. | We enforce the preventative guardrails at the account level and restrict the changes to Firewall rules, CloudTrail rules, CloudWatch logging rules and others. |
| Network Sniffing attempts | We monitor the network sniffing attempts | We enable network security controls such as Network firewall, VPC Flow logs, VPC traffic mirroring, Amazon GuardDuty and such |
| Non-Standard Ports usage | We monitor the port changes to the servers | We configure the preventative and detective guardrails at the account level to restrict and monitor the ports |
| Proxy anomalies | ||
| Resource Hijacking | We monitor the AWS resource usage for anomalies and known patterns such as crypto mining | We use Amazon GuardDuty to monitor the suspicious activity and alert |
| Subvert Trust Controls | We use only the trusted resources | We use methods such as code signing using AWS Signer, certificate-based authentication. |
| Valid/Invalid Accounts | We monitor the account and role usage. | We use principle of least privilege and use tools such as IAM access analyser to monitor the AWS account usage and take corrective actions. |
Footnote
We acknowledge and sincerely thank Mr. Ratan Jyoti, CISO, Ujjivan SFB for his inputs to "Drivers for Security Operation and controls (SOC) and SIEM" and "Core Events for SIEM platform" sections of the blog.
DISCLAIMER: The information in this article about software testing techniques and best practices is provided for general educational purposes only. The author is not an attorney and this article does not constitute legal advice. Readers should consult an attorney if they need legal advice about software testing or any other topic covered in this article. The author and publisher disclaim any liability arising from reliance on the information contained herein. The software/ tools and brand names mentioned in this article are trademarks or registered trademarks of their respective owner entities. The author is not affiliated with or sponsored by any of the third party software owners mentioned. The reference to third party software is for informational purposes only and does not constitute an endorsement or recommendation.
