Skip to content
This repository has been archived by the owner on Jul 26, 2023. It is now read-only.

Commit

Permalink
Merge pull request #53 from wazuh/feature-51-prod-ready-cluster
Browse files Browse the repository at this point in the history
New production-ready environment
  • Loading branch information
Jose M. Garcia authored Apr 16, 2020
2 parents 58d2f39 + 3009c7f commit 515982d
Show file tree
Hide file tree
Showing 28 changed files with 3,023 additions and 106 deletions.
106 changes: 13 additions & 93 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,105 +5,25 @@
[![Documentation](https://img.shields.io/badge/docs-view-green.svg)](https://documentation.wazuh.com)
[![Web](https://img.shields.io/badge/web-view-green.svg)](https://wazuh.com)

This repository includes the template and scripts to set up an environment that includes:
This repository contains CloudFormation templates and provision scripts to deploy both a Wazuh production-ready environment and a Wazuh demo environment in Amazon Web Services (AWS):

## Production-ready environment:

* A VPC with two subnets, one for Wazuh servers, and another for Elastic Stack
* Wazuh managers cluster with two nodes, a master and a worker
* An Elasticsearch cluster with a minimum of 3 data nodes, auto-scalable to a maximum of 6 nodes
* A Kibana node that includes a local elasticsearch client node, and an Nginx for HTTP basic authentication
* Wazuh servers seat behind an internet-facing load balancer for agents to communicate with the cluster
* Kibana server seats behind an internet facing load balancer, that optionally loads an SSL Certificate for HTTPS
* A Splunk Indexer instance with a Splunk app for Wazuh installed on it.
* Six Wazuh agents installed on different operating systems: Red Hat 7, CentOS 7, Ubuntu, Debian, Amazon Linux and Windows.

## Elasticsearch cluster configuration

Elasticsearch data nodes are deployed as part of an auto scaling group, that scales based on CPU usage. Minimum number of nodes is 3, and maximum is 6.

Elasticsearch instance types can be chosen from:

* i3.large
* i3.xlarge
* i3.2xlarge
* t2.large
* t2.medium

These instance types are recommended due to Elasticsearch disk requirements. Ephemeral disks are used for data storage.

None of these instances are directly accessible from the Internet, although they can be reached jumping through the Kibana system, that has a public SSH service.

## Kibana server configuration

Kibana server runs an instance of Elasticsearch (acting as a client node), an instance of Kibana (with Wazuh plugin installed and configured), and an instance of Nginx (used to provide SSL encryption and basic HTTP authentication).

Kibana instance types can be chosen from:

* m5.large
* m5.xlarge
* m5.2xlarge
* t2.large
* t2.medium
* r5.large

These instance types are recommended due to Kibana and Elasticsearch memory requirements.

In addition, the Kibana server takes care of:

* Setting up wazuh-alerts template in Elasticsearch
* Setting default index-pattern to wazuh-alerts
* Setting default time-picker to 24 hours

Kibana server is reachable from the Internet, directly via its own Elastic IP, or through an internet-facing load balancer. The load balancer can be used, optionally, to add a valid Amazon SSL Certificate for HTTPS communications.

## Wazuh cluster configuration

The Wazuh cluster deployed has one master node (providing API and registration server) and one worker node.

Wazuh instance types can be chosen from:

* m5.large
* m5.xlarge
* m5.2xlarge
* t2.micro
* t2.medium
* t2.large

These instance types are recommended for the managers, as they provide enough memory for Wazuh components.
* Route53 DNS records for the loadbalancer, Wazuh and Elastic Stack nodes (optional).

The Wazuh API, running on Wazuh master node, is automatically configured to use HTTPS protocol.
## Demo environment:

The Wazuh registration service (authd), running on Wazuh master node, is configured not to use source IP addresses. We assume that agents will connect through the Internet, and most likely several will use the same source IP (sitting behind a NAT). This service is configured automatically to require password authentication for new agents registration.

Filebeat runs on both the Wazuh master node and the worker node, reading alerts and forwarding those to Elasticsearch nodes via the internal load balancer.

New agents can make use of the Wazuh master public Elastic IP address for registration.

Once registered, new agents can connect to the Wazuh cluster, via TCP, using the load balancer public IP address.

## Optional DNS records

A parent domain (e.g. mycompany.com) and subdomain (e.g. wazuh) can be specified. In this example, this is what would be used for communications:

* wazuh.mycompany.com: domain name for access to Kibana WUI (via HTTPS). It also provides access via SSH (jumpbox to servers).
* registration.wazuh.mycompany.com: domain name for agents registration.
* data.wazuh.mycompany.com: domain name for agents communication with the cluster.

An example of the installation of a new agent, on a Windows system (automatically registered and configured) using an MSI package would be:

wazuh-agent-3.12.0-1.msi /q ADDRESS=“wazuh.mycompany.com” AUTHD_SERVER=“registration.wazuh.mycompany.com” PASSWORD=“mypassword” AGENT_NAME=“myhostname” PROTOCOL=“TCP”

An example of the registration of a new agent on a Linux system would be:

/var/ossec/bin/agent-auth -m registration.wazuh.mycompany.com -P mypassword -A myhostname

Then, on the linux agent, the /var/ossec/etc/ossec.conf would include the configuration to connect to the managers:

<server>
<address>data.wazuh.mycompany.com</address>
<port>1514</port>
<protocol>tcp</protocol>
</server>

## AWS environment created by cloud formation template

![wazuh_template](images/wazuh_template-designer.png)
* A VPC with two subnets, one for Wazuh servers, and another for Elastic Stack
* Wazuh managers cluster with two nodes, a master and a worker
* An Elasticsearch cluster with a minimum of 3 data nodes, auto-scalable to a maximum of 6 nodes
* A Kibana node that includes a local elasticsearch client node, and an Nginx for HTTP basic authentication
* Wazuh servers seat behind an internet-facing load balancer for agents to communicate with the cluster
* Kibana server seats behind an internet facing load balancer, that optionally loads an SSL Certificate for HTTPS
* A Splunk Indexer instance with a Splunk app for Wazuh installed on it.
* Six Wazuh agents installed on different operating systems: Red Hat 7, CentOS 7, Ubuntu, Debian, Amazon Linux and Windows.
File renamed without changes.
109 changes: 109 additions & 0 deletions demo/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Wazuh for Amazon AWS Cloudformation

[![Slack](https://img.shields.io/badge/slack-join-blue.svg)](https://goo.gl/forms/M2AoZC4b2R9A9Zy12)
[![Email](https://img.shields.io/badge/email-join-blue.svg)](https://groups.google.com/forum/#!forum/wazuh)
[![Documentation](https://img.shields.io/badge/docs-view-green.svg)](https://documentation.wazuh.com)
[![Web](https://img.shields.io/badge/web-view-green.svg)](https://wazuh.com)

The template and scripts here set up an environment that includes:

* A VPC with two subnets, one for Wazuh servers, and another for Elastic Stack
* Wazuh managers cluster with two nodes, a master and a worker
* An Elasticsearch cluster with a minimum of 3 data nodes, auto-scalable to a maximum of 6 nodes
* A Kibana node that includes a local elasticsearch client node, and an Nginx for HTTP basic authentication
* Wazuh servers seat behind an internet-facing load balancer for agents to communicate with the cluster
* Kibana server seats behind an internet facing load balancer, that optionally loads an SSL Certificate for HTTPS
* A Splunk Indexer instance with a Splunk app for Wazuh installed on it.
* Six Wazuh agents installed on different operating systems: Red Hat 7, CentOS 7, Ubuntu, Debian, Amazon Linux and Windows.

## Elasticsearch cluster configuration

Elasticsearch data nodes are deployed as part of an auto scaling group, that scales based on CPU usage. Minimum number of nodes is 3, and maximum is 6.

Elasticsearch instance types can be chosen from:

* i3.large
* i3.xlarge
* i3.2xlarge
* t2.large
* t2.medium

These instance types are recommended due to Elasticsearch disk requirements. Ephemeral disks are used for data storage.

None of these instances are directly accessible from the Internet, although they can be reached jumping through the Kibana system, that has a public SSH service.

## Kibana server configuration

Kibana server runs an instance of Elasticsearch (acting as a client node), an instance of Kibana (with Wazuh plugin installed and configured), and an instance of Nginx (used to provide SSL encryption and basic HTTP authentication).

Kibana instance types can be chosen from:

* m5.large
* m5.xlarge
* m5.2xlarge
* t2.large
* t2.medium
* r5.large

These instance types are recommended due to Kibana and Elasticsearch memory requirements.

In addition, the Kibana server takes care of:

* Setting up wazuh-alerts template in Elasticsearch
* Setting default index-pattern to wazuh-alerts
* Setting default time-picker to 24 hours

Kibana server is reachable from the Internet, directly via its own Elastic IP, or through an internet-facing load balancer. The load balancer can be used, optionally, to add a valid Amazon SSL Certificate for HTTPS communications.

## Wazuh cluster configuration

The Wazuh cluster deployed has one master node (providing API and registration server) and one worker node.

Wazuh instance types can be chosen from:

* m5.large
* m5.xlarge
* m5.2xlarge
* t2.micro
* t2.medium
* t2.large

These instance types are recommended for the managers, as they provide enough memory for Wazuh components.

The Wazuh API, running on Wazuh master node, is automatically configured to use HTTPS protocol.

The Wazuh registration service (authd), running on Wazuh master node, is configured not to use source IP addresses. We assume that agents will connect through the Internet, and most likely several will use the same source IP (sitting behind a NAT). This service is configured automatically to require password authentication for new agents registration.

Filebeat runs on both the Wazuh master node and the worker node, reading alerts and forwarding those to Elasticsearch nodes via the internal load balancer.

New agents can make use of the Wazuh master public Elastic IP address for registration.

Once registered, new agents can connect to the Wazuh cluster, via TCP, using the load balancer public IP address.

## Optional DNS records

A parent domain (e.g. mycompany.com) and subdomain (e.g. wazuh) can be specified. In this example, this is what would be used for communications:

* wazuh.mycompany.com: domain name for access to Kibana WUI (via HTTPS). It also provides access via SSH (jumpbox to servers).
* registration.wazuh.mycompany.com: domain name for agents registration.
* data.wazuh.mycompany.com: domain name for agents communication with the cluster.

An example of the installation of a new agent, on a Windows system (automatically registered and configured) using an MSI package would be:

wazuh-agent-3.12.2-1.msi /q ADDRESS=“wazuh.mycompany.com” AUTHD_SERVER=“registration.wazuh.mycompany.com” PASSWORD=“mypassword” AGENT_NAME=“myhostname” PROTOCOL=“TCP”

An example of the registration of a new agent on a Linux system would be:

/var/ossec/bin/agent-auth -m registration.wazuh.mycompany.com -P mypassword -A myhostname

Then, on the linux agent, the /var/ossec/etc/ossec.conf would include the configuration to connect to the managers:

<server>
<address>data.wazuh.mycompany.com</address>
<port>1514</port>
<protocol>tcp</protocol>
</server>

## AWS demo environment diagram

![wazuh_template](images/wazuh_template-designer.png)
2 changes: 1 addition & 1 deletion deploy.sh → demo/deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ fi
# Uploading template to S3
aws s3 cp $TEMPLATE_FILE s3://$BUCKET_NAME
# Getting the template URL
URL="https://demo-cloudformation-templates.s3-us-west-1.amazonaws.com/wazuh_template.yml"
URL="https://$BUCKET_NAME.s3-us-west-1.amazonaws.com/wazuh_template.yml"
echo "Template URL: $URL"

aws cloudformation create-stack --stack-name ${STACK_NAME} --template-url $URL --parameters file://$PARAMS_FILE --capabilities CAPABILITY_IAM --tags Key=service_name,Value=demo_info
Expand Down
File renamed without changes.
13 changes: 13 additions & 0 deletions elastic/wazuh_cf_kibana.sh → demo/elastic/wazuh_cf_kibana.sh
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,18 @@ install_plugin(){

}


optimize_kibana(){
systemctl stop kibana
echo "Optimizing app" >> /tmp/deploy.log
cd /usr/share/kibana
sudo -u kibana /usr/share/kibana/node/bin/node --no-warnings --max-old-space-size=2048 --max-http-header-size=65536 /usr/share/kibana/src/cli --optimize
cd /tmp
echo "App installed!" >> /tmp/deploy.log
systemctl start kibana
}


add_api(){
echo "Adding Wazuh API" >> /tmp/deploy.log
sed -ie '/- default:/,+4d' /usr/share/kibana/optimize/wazuh/config/wazuh.yml
Expand Down Expand Up @@ -430,6 +442,7 @@ main(){
get_plugin_url
install_plugin
enable_kibana
optimize_kibana
start_kibana
sleep 60
add_api
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
24 changes: 12 additions & 12 deletions wazuh_template.yml → demo/wazuh_template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -726,7 +726,7 @@ Resources:
group: root
/tmp/wazuh_cf_bootstrap_elastic.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/elastic/wazuh_cf_bootstrap_elastic.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/elastic/wazuh_cf_bootstrap_elastic.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -805,7 +805,7 @@ Resources:
group: root
/tmp/wazuh_cf_elastic.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/elastic/wazuh_cf_elastic.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/elastic/wazuh_cf_elastic.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -884,7 +884,7 @@ Resources:
group: root
/tmp/wazuh_cf_elastic.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/elastic/wazuh_cf_elastic.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/elastic/wazuh_cf_elastic.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -1111,7 +1111,7 @@ Resources:
group: root
/tmp/wazuh_cf_kibana.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/elastic/wazuh_cf_kibana.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/elastic/wazuh_cf_kibana.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -1188,7 +1188,7 @@ Resources:
group: root
/tmp/wazuh_cf_splunk.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/splunk/wazuh_cf_splunk.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/splunk/wazuh_cf_splunk.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -1282,7 +1282,7 @@ Resources:
group: root
/tmp/wazuh_cf_master.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/wazuh/cluster/wazuh_cf_master.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/wazuh/cluster/wazuh_cf_master.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -1371,7 +1371,7 @@ Resources:
group: root
/tmp/wazuh_cf_worker.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/wazuh/cluster/wazuh_cf_worker.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/wazuh/cluster/wazuh_cf_worker.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -1452,7 +1452,7 @@ Resources:
group: root
/tmp/wazuh_agent_amazon.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/wazuh/agents/wazuh_agent_amazon.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/wazuh/agents/wazuh_agent_amazon.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -1533,7 +1533,7 @@ Resources:
group: root
/tmp/wazuh_agent_centos.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/wazuh/agents/wazuh_agent_centos.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/wazuh/agents/wazuh_agent_centos.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -1707,7 +1707,7 @@ Resources:
group: root
/tmp/wazuh_agent_debian.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/wazuh/agents/wazuh_cf_agent_debian.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/wazuh/agents/wazuh_cf_agent_debian.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -1795,7 +1795,7 @@ Resources:
group: root
/tmp/wazuh_agent_ubuntu.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/wazuh/agents/wazuh_cf_agent_ubuntu.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/wazuh/agents/wazuh_cf_agent_ubuntu.sh
mode: '000700'
owner: root
group: root
Expand Down Expand Up @@ -1883,7 +1883,7 @@ Resources:
group: root
/tmp/wazuh_agent_rhel.sh:
source: !Sub >-
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/wazuh/agents/wazuh_agent_rhel.sh
https://raw.githubusercontent.com/wazuh/wazuh-cloudformation/v3.12.2_7.6.2/demo/wazuh/agents/wazuh_agent_rhel.sh
mode: '000700'
owner: root
group: root
Expand Down
Loading

0 comments on commit 515982d

Please sign in to comment.