InfoSec Zanshinhttps://www.infoseczanshin.com/2019-12-30T19:51:00-03:00Using OpenVPN Access Server to Access AWS VPCs2019-12-30T19:51:00-03:002019-12-30T19:51:00-03:00Alexandre Sieiratag:www.infoseczanshin.com,2019-12-30:/using-openvpn-access-server-to-access-aws-vpcs.html<p>Establishing a client-to-site VPN with AWS VPCs using auto-renewing Let's Encrypt certificates</p><p>If you use AWS for anything non-trivial, you are likely using
<a href="https://aws.amazon.com/documentation/vpc/">VPCs</a> and keeping some non-public subnets for
resources you don't want to be accessible to the open Internet. </p>
<p>Usually when this is done, a
<a href="https://aws.amazon.com/blogs/security/how-to-record-ssh-sessions-established-through-a-bastion-host/">bastion host</a> is set up on a VPC public subnet and accessed via
SSH to reach those internal instances or resources, through secondary SSH connections or
SSH tunnels. That is a good and simple solution that will work well for a simple environment
with few internal services and few users that need to access it.</p>
<p>However, I would like to explore an alternative: using
<a href="https://openvpn.net/index.php/access-server/on-amazon-cloud.html">OpenVPN Access Server</a> to
create a client-to-site VPN connection to the VPC. This approach is really easy to implement
and has some pretty interesting benefits:</p>
<ul>
<li>
<p>Easy set up of two-factor authentication with
<a href="https://en.wikipedia.org/wiki/Google_Authenticator">Google Authenticator</a> or similar TOTP
solutions. This is something you could also achieve with the bastion host as well using some
<a href="https://sysconfig.org.uk/two-factor-authentication-with-ssh.html">PAM magic</a> if you really
want to, by the way. </p>
</li>
<li>
<p>Use of different IP address ranges to users of different groups, which allows different
<a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_SecurityGroups.html">Security Group</a>
inbound rules on assets for internal resources based on who the authenticated VPN users are.</p>
</li>
<li>
<p>A web interface for your users that simplifies the task of downloading the VPN client with
the proper connection profile, which is far more manageable for users that are not of a
strong technical background.</p>
</li>
<li>
<p>Commercial support from the team at
<a href="https://openvpn.net/index.php/login.html">OpenVPN</a>.</p>
</li>
</ul>
<p>One of the downsides is that this solution will require you to buy
<a href="https://openvpn.net/index.php/access-server/pricing.html">licenses</a> for the OpenVPN
Access Server if you plan to have more than two client connections at a time. But I find them
to be reasonably priced.</p>
<p>This post was recently updated based on a fresh install using OpenVPN Access Server 2.7.5 and is
meant as a rough but coherent guide to the external references I used to get this working.
Hopefully it will save someone some time.</p>
<h2>OpenVPN Access Server Instance</h2>
<p>First, go to the <a href="https://aws.amazon.com/marketplace/">AWS Marketplace</a> and find the
<a href="https://aws.amazon.com/marketplace/pp/B00MI40CAE">BYOL AMI for OpenVPN Access Server</a>. Then,
follow their
<a href="https://docs.openvpn.net/how-to-tutorialsguides/virtual-platforms/amazon-web-services-ec2-tiered-appliance-quick-start-guide/">Quick Start Guide</a>
to perform the initial configuration.</p>
<p>A few things to keep in mind:</p>
<ul>
<li>
<p>Make sure you install this on a public subnet of your VPC;</p>
</li>
<li>
<p>Assign the OpenVPN server EC2 instance an
<a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-eips.html">elastic IP address</a>
so that if you stop and restart it the IP address won't change.</p>
</li>
</ul>
<h2>Updates, Updates and More Updates</h2>
<p>The first thing you'll want to do after you have finished the SSH questions and are logged
into the instance is to fully update it and then reboot it by running the following commands:</p>
<div class="bash highlight"><pre><span></span><code>sudo su -
apt-get update
apt-get dist-upgrade
shutdown -r now
</code></pre></div>
<p>Please don't skip the reboot, so that all updates actually take effect, such as updates to the
kernel and to libraries that are currently being used by running processes.</p>
<p>After you reboot, the default installation will automatically run the
<a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html">Systems Manager Agent</a>,
so if you create a role with the proper permissions you should be able to log in using
<a href="https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager.html">Session Manager</a>
and close access via SSH entirely.</p>
<h2>Web Server Certificates</h2>
<p>Now let's set up the OpenVPN Access Server web server so use <a href="https://letsencrypt.org/">Let's Encrypt</a>
certificates with automated renewal.</p>
<p>First, as <code>root</code>, <a href="https://certbot.eff.org/lets-encrypt/ubuntubionic-other">install certbot</a> using DNS validation
through Route53:</p>
<div class="bash highlight"><pre><span></span><code>apt-get install software-properties-common
add-apt-repository universe
add-apt-repository ppa:certbot/certbot
apt-get update
apt-get install python3-certbot-dns-route53
</code></pre></div>
<p>Then make sure you append a new policy to the OpenVPN instance role with the
<a href="https://certbot-dns-route53.readthedocs.io/en/stable/">necessary permissions</a>, replacing <code>YOURHOSTEDZONEID</code> with
the correct zone ID:</p>
<div class="json highlight"><pre><span></span><code><span class="p">{</span>
<span class="nt">"Version"</span><span class="p">:</span> <span class="s2">"2012-10-17"</span><span class="p">,</span>
<span class="nt">"Id"</span><span class="p">:</span> <span class="s2">"certbot-dns-route53 sample policy"</span><span class="p">,</span>
<span class="nt">"Statement"</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="nt">"Effect"</span><span class="p">:</span> <span class="s2">"Allow"</span><span class="p">,</span>
<span class="nt">"Action"</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">"route53:ListHostedZones"</span><span class="p">,</span>
<span class="s2">"route53:GetChange"</span>
<span class="p">],</span>
<span class="nt">"Resource"</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">"*"</span>
<span class="p">]</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="nt">"Effect"</span> <span class="p">:</span> <span class="s2">"Allow"</span><span class="p">,</span>
<span class="nt">"Action"</span> <span class="p">:</span> <span class="p">[</span>
<span class="s2">"route53:ChangeResourceRecordSets"</span>
<span class="p">],</span>
<span class="nt">"Resource"</span> <span class="p">:</span> <span class="p">[</span>
<span class="s2">"arn:aws:route53:::hostedzone/YOURHOSTEDZONEID"</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
</code></pre></div>
<p>Take this time to also create an <code>A</code> record set in Route53 that maps the hostname of your OpenVPN server to the
Elastic IP address you assigned to it in the previous section.</p>
<p>In order to be ready for automated renewal later on, let's do this by creating three scripts on specific paths that
certbot uses when creating and loading certs (pre, deploy and post hooks) that I adapted from
<a href="https://www.sideras.net/lets-encrypt-https-certificates-for-openvpn-as-access-server/">elsewhere</a>:</p>
<div class="bash highlight"><pre><span></span><code>cat > /etc/letsencrypt/renewal-hooks/pre/stop_openvpn.sh <span class="s"><< EOF</span>
<span class="s">#!/bin/sh</span>
<span class="s">/usr/local/openvpn_as/scripts/sacli stop</span>
<span class="s">EOF</span>
chmod a+x /etc/letsencrypt/renewal-hooks/pre/stop_openvpn.sh
cat > /etc/letsencrypt/renewal-hooks/post/start_openvpn.sh <span class="s"><< EOF</span>
<span class="s">#!/bin/sh</span>
<span class="s">/usr/local/openvpn_as/scripts/sacli start</span>
<span class="s">EOF</span>
chmod a+x /etc/letsencrypt/renewal-hooks/post/start_openvpn.sh
cat > /etc/letsencrypt/renewal-hooks/deploy/load_openvpn.sh <span class="s"><< EOF</span>
<span class="s">#!/bin/sh</span>
<span class="s">/usr/local/openvpn_as/scripts/confdba -mk cs.ca_bundle -v "`cat /etc/letsencrypt/live/openvpn.example.com/fullchain.pem`"</span>
<span class="s">/usr/local/openvpn_as/scripts/confdba -mk cs.priv_key -v "`cat /etc/letsencrypt/live/openvpn.example.com/privkey.pem`" > /dev/null</span>
<span class="s">/usr/local/openvpn_as/scripts/confdba -mk cs.cert -v "`cat /etc/letsencrypt/live/openvpn.example.com/cert.pem`"</span>
<span class="s">EOF</span>
chmod a+x /etc/letsencrypt/renewal-hooks/deploy/load_openvpn.sh
</code></pre></div>
<p>Please make sure you replace <code>openvpn.example.com</code> in the code above with the actual hostname you chose for your
OpenVPN server.</p>
<p>Then, request a certbot certificate by issuing the following command and following the on-screen instructions:</p>
<div class="bash highlight"><pre><span></span><code>certbot certonly --dns-route53 -d openvpn.example.com
</code></pre></div>
<p>At this point, as soon as the OpenVPN service comes back up it should be using the newly generated certificate.</p>
<p>Finally, create a regular task executed twice a day by running <code>crontab -e</code> as <code>root</code> and entering the following new entry:</p>
<div class="highlight"><pre><span></span><code><span class="err">0 1,13 * * * /usr/bin/certbot renew --quiet</span>
</code></pre></div>
<p>Please choose an arbitrary number between 1 and 60 and replace the first <code>0</code> in the above line. Also choose
different hours (instead of <code>1,13</code>) for executing this. If lots of people configure their cron jobs to hit
Let's Encrypt at exactly the same times we might end up with an accidental DDoS.</p>
<p>At this point, you should be able to access the admin interface (by default in our example it would be
<code>https://openvpn.example.com:943/admin</code>), and not get a warning about an invalid certificate. Just remove the
<code>/admin</code> from the URL and you'll land on the
<a href="https://openvpn.net/index.php/access-server/section-faq-openvpn-as/client-configuration.html">user-focused page</a>
instead, where they can log in to download the client software binaries and profile configuration, and also scan
Google Authenticator QR codes.</p>
<p>Go into the admin interface and perform the configuration that best suits your needs. We'll discuss two
particularly interesting aspects of the configuration next.</p>
<h2>Routing</h2>
<p>You may choose to have OpenVPN use routing instead of NAT when it forwards traffic from the connected clients
into the VPC environment. If you do that, you can use its ability to assign different users or groups of users
IP addresses in different ranges. This, in turn, allows you to configure Security Group rules inside the VPC
that will only allow certain users or groups of users to communicate with specific services.</p>
<p>If you do that, however, keep in mind that you'll need to take some additional steps:</p>
<ul>
<li>
<p>Ensure that the IP address ranges you assign VPN clients are <em>outside</em> the CIDR range for the entire VPC.
So if your VPC has addresses on 172.16.0.0/16, you can't assign any part of that address space to VPN clients
even if there are no VPC subnets currently using them.</p>
</li>
<li>
<p>You'll need to update your VPC routing configuration. For each CIDR of addresses that can be assigned to VPN
clients, <a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Route_Tables.html#AddRemoveRoutes">add a new route</a>
with that CIDR as the <code>Destination</code> and the OpenVPN server instance as the <code>Target</code>. </p>
</li>
<li>
<p><a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCheck">Disable the source/destination check</a>
on the OpenVPN Access Server instance so that it can "answer for" other ranges of IP address in the AWS network
stack.</p>
</li>
</ul>
<h2>Google Authenticator</h2>
<p>If everything else is working at this point, requiring users to have two-factor authentication using
Google Authenticator should be pretty easy by now. First, on the admin interface go to <code>Configuration</code> →
<code>Client Settings</code> and check the box that says <code>Require that users provide a Google Authenticator one-time
password for every VPN login</code>.</p>
<p>What it took me a while to figure out was how to enroll users. Basically what you do is create the user
normally in the admin interface in <code>User Management</code> → <code>User Permissions</code>, and assign the user a password.</p>
<p>Then, the end user must log in to the non-admin web interface (<code>https://openvpn.example.com</code> in our
example). In this page, they'll the given the choice to download a connection client or a profile configuration they
can import into an existing client. But they will also be displayed the QR code they'll scan using
Google Authenticator. Once they do that, everything should be good to go.</p>
<h2>CloudWatch Logs Agent</h2>
<p>If you are using System Manager, then at this point you should have the
<a href="https://aws.amazon.com/pt/blogs/aws/new-amazon-cloudwatch-agent-with-aws-systems-manager-integration-unified-metrics-log-collection-for-linux-windows/">Unified CloudWatch Agent</a>
installed at the machine. We can use it to ensure the OpenVPN Server logs are centralized and easy to query if necessary,
and also protected from any attackers that might gain access to the server.</p>
<p>Firstly, I recommend assigning the machine its proper hostname by issuing the following command as <code>root</code>:</p>
<div class="bash highlight"><pre><span></span><code>hostname openvpn.example.com
</code></pre></div>
<p>Then, make sure you edit <code>/etc/hostname</code> and <code>/etc/hosts</code> to make sure they also match the the full hostname.</p>
<p>Finally, you can install <code>collectd</code> and then execute the agent configuration wizard by running the following command as root:</p>
<div class="bash highlight"><pre><span></span><code>apt-get install collectd
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
</code></pre></div>
<p>The options will be saved at <code>/opt/aws/amazon-cloudwatch-agent/bin/config.json</code> which you can always go and edit
manually later if needed. After editing this file you can run
<code>amazon-cloudwatch-agent-ctl -m ec2 -a fetch-config -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json -s</code>
to force the agent to reload the configuration and restart.</p>
<p>A few pointers:
* Make sure you include <code>/var/log/openvpnas.log</code>, <code>/var/log/syslog</code> and <code>/var/log/auth.log</code> to the log
collection.</p>
<ul>
<li>
<p>When asked what the log stream name should be, I recommend you use the newly-configured machine name
for clarity.</p>
</li>
<li>
<p>Set the service user as <code>cwagent</code>.</p>
</li>
</ul>
<p>Then, update <code>/etc/rsyslog.conf</code> as per <a href="http://serverfault.com/a/592120">these instructions</a> and
restart the rsyslog service:</p>
<div class="bash highlight"><pre><span></span><code>service rsyslog restart
</code></pre></div>
<p>You should now see the new log groups on the AWS Console.</p>
<p>I would also like to suggest that you create a new file <code>/etc/logrotate.d/openvpnas</code> with the following
content:</p>
<div class="highlight"><pre><span></span><code><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="nb">log</span><span class="o">/</span><span class="n">openvpnas</span><span class="o">.</span><span class="n">log</span> <span class="p">{</span>
<span class="n">rotate</span> <span class="mi">90</span>
<span class="n">daily</span>
<span class="n">compress</span>
<span class="n">missingok</span>
<span class="n">notifempty</span>
<span class="p">}</span>
</code></pre></div>
<h2>Additional Recommendations</h2>
<p>A few security-minded tips that I would recommend you implement:</p>
<ul>
<li>
<p>On the web admin interface, go to <code>Configuration</code> → <code>TLS Settings</code> and select <code>TLS 1.2</code> as the minimum
for both the web server and OpenVPN server, as per
<a href="https://www.owasp.org/index.php/Transport_Layer_Protection_Cheat_Sheet#Rule_-_Only_Support_Strong_Protocols">industry best practice</a>.</p>
</li>
<li>
<p>Harden the operating system and make sure to keep install security updates as they become available.</p>
</li>
<li>
<p>Secure the <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html">AWS instance metadata service</a>
by preventing the <code>openvpn</code> and other users that don't need it from accessing it:</p>
</li>
</ul>
<div class="bash highlight"><pre><span></span><code>sudo apt-get install iptables-persistent
sudo iptables -A OUTPUT -d <span class="m">169</span>.254.169.254 -p tcp -m owner --uid-owner cwagent -j ACCEPT
sudo iptables -A OUTPUT -d <span class="m">169</span>.254.169.254 -p tcp -m owner --uid-owner ssm-user -j ACCEPT
sudo iptables -A OUTPUT -d <span class="m">169</span>.254.169.254 -p tcp -m owner --uid-owner root -j ACCEPT
sudo iptables -A OUTPUT -d <span class="m">169</span>.254.169.254 -p tcp -j LOG --log-prefix <span class="s2">"iptables-metadata-dropped"</span> --log-level <span class="m">4</span>
sudo iptables -A OUTPUT -d <span class="m">169</span>.254.169.254 -p tcp -j REJECT
sudo /etc/init.d/netfilter-persistent save
</code></pre></div>
<p>You can remove the line allowing <code>root</code> if you are not using the certbot Route53 automated certificate renewal, or maybe
even create a specific user to run that task.</p>Learning from the July 2019 Capital One Breach2019-07-30T12:00:00-03:002019-07-30T12:00:00-03:00Alexandre Sieiratag:www.infoseczanshin.com,2019-07-30:/learning-from-the-july-2019-capital-one-breach.html<p>Practical Advice to Prevent IAM Credential Abuse</p><p>News broke today of <a href="https://www.washingtonpost.com/national-security/capital-one-data-breach-compromises-tens-of-millions-of-credit-card-applications-fbi-says/2019/07/29/72114cc2-b243-11e9-8f6c-7828e68cb15f_story.html?utm_term=.f5b3ccbc7616">breach involving Capital One</a>, involving the theft of data stored in non-public S3 buckets through a multi-step targeted attack. </p>
<p>The first thing I want to make clear is that I sympathize with the Capital One security and operations teams at this difficult time. Capital One is a well-known innovator in cloud security, has very competent people dedicated to this and has even developed high quality open source solutions such as <a href="https://cloudcustodian.io/">Cloud Custodian</a> that benefit the entire community. No security is perfect, and if you are a big enough target get used to the idea that being breached is a question of "when" not "if".</p>
<p>I sincerely hope that this incident is not blown out of proportion and does not cause a slow down of their strategy of being a cloud-first bank by next year, as they announced on stage at <a href="https://www.infoseczanshin.com/highlights-from-aws-reinforce-2019.html">AWS re:Inforce</a>. In any case, I think that an organization as heavily invested into cloud security as Capital One being breached is a good reminder that security is hard. Also, that even though the cloud offers many advantages in applying security over on-premises environments, it is still relatively new and evolving technology and there's a non-trivial learning curve.</p>
<p>You can see the <a href="#resources">Resources</a> section below for detailed write-ups, and the <a href="https://regmedia.co.uk/2019/07/29/capital_one_paige_thompson.pdf">affidavit</a> describing what is known so far about this case. Reading through this and discussing this with others, I'd like to document a few practical guidelines and mitigations that might help prevent similar attacks in the future.</p>
<h2>Least Privilege</h2>
<p>The attack involved the attacker obtaining temporary credentials associated with an IAM role referenced as <code>*****-WAF-Role</code> in the <a href="https://regmedia.co.uk/2019/07/29/capital_one_paige_thompson.pdf">affidavit</a>. A big part of the problem seems to be that this role had excessive privileges: </p>
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">For service roles, remove s3:ListAllMyBuckets and similar List/Describe privs. Code normally knows what buckets it needs to work with, and does not need to list them.</p>— Scott Piper (@0xdabbad00) <a href="https://twitter.com/0xdabbad00/status/1156011925542584320?ref_src=twsrc%5Etfw">July 30, 2019</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Being able to use the unnecessary <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/RESTServiceGET.html">list buckets</a> privilege granted to the role allowed the attacker to find out and explore multiple buckets until they found interesting data. Had that privilege not been granted to the role, and assuming they didn't have insider knowledge of the infrastructure, the attacker might have been slowed down or even prevented from finding buckets with sensitive data entirely.</p>
<p>The same IAM role credentials were then used to list and download data from the buckets apparently using the AWS CLI <a href="https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html"><code>s3 sync</code></a> command, which actually uses multiple bucket list and object read operations to do its work.</p>
<p>The name indicates that this role was being used by a Web Application Firewall solution. So presumably the S3 privileges were either meant to allow it to access websites published via S3 or to write logs or configuration backups to S3. It wouldn't surprise me if someone simply granted access to all buckets or even to all S3 operations during deployment either manually or using an <a href="https://www.tripwire.com/state-of-security/security-data-protection/cloud/aws-system-manager-default-permissions/">overly permissive AWS Managed Policy</a> in the same role.</p>
<p>So two lessons here:</p>
<ul>
<li>
<p><strong>Ensure your policies allow as few Actions and Resources as possible!</strong> Avoid using <code>*</code> like the plague, and always provide an explicit list of Actions and Resources in your policies.</p>
</li>
<li>
<p>Don't trust AWS Managed Policies blindly. Review them and, when necessary, create more specific and more secure versions for your own use. Particularly when they violate the first lesson.</p>
</li>
</ul>
<p>If the WAF role in this incident only had access to writing new objects into a single bucket containing WAF logs, the impact from this intrusion might have been minimized.</p>
<h2>Control Permission Use Context with Policy Conditions</h2>
<p>The other thing that caught my attention on this breach is that the attacker was able to get <a href="https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html">temporary credentials from an IAM role</a> assigned to a resource on Capital One's VPC and then use them from an external computer under their control to download data from the S3 buckets. </p>
<p>On an on-premises environment, a "private" FTP or NFS server would be on an internal network not accessible from the Internet, so the attacker would not only need credentials but also network access in order to use them. That's not how S3 works, however. Even non-public S3 buckets can be accessed by default from anywhere in the world by someone holding the right credentials. </p>
<p>Turns out, though, that you can implement something similar in the case of S3 buckets. A very infrequently used security measure for S3 buckets is to limit the access to API requests originated on a specific VPC, <a href="https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints-s3.html">VPC endpoint</a> of IP address ranges by using <a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/amazon-s3-policy-keys.html">policy conditions</a>.</p>
<p>So at a minimum, if you have an IAM role granting access to a particular S3 bucket, and that role is only meant to be used by on-VPC resources such as EC2 instances, you could make it so that the access is only granted if the VPC or VPC endpoint is the one you expect.</p>
<p>Even better, if a particular bucket will only be used by resources on your VPC, you could add a <a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies-vpc-endpoint.html">statement to the bucket policy denying any access attempt that does not originate from the specific VPC or VPC endpoints you control</a> to its bucket policy. This would ensure that even if excessive privileges are granted to users or roles, the Deny statement will take precedence and no access to that bucket from outside of your VPC will ever happen. This would be the closest equivalent to the "FTP or NFS server in the internal network" analogy on an on-premises environment.</p>
<p>Either of those measures might have prevented the attacker from using the role's temporary credentials to access those non-public buckets from a computer outside of the company VPC to download the data. The attacker would have then needed to take additional steps, such as compromising or creating a new resource on the Capital One VPC with sufficient outbound network permissions, in order to exfiltrate the data. This might have created the conditions that allowed the intrusion to be detected before it was successful.</p>
<p>This technique can be generalized to other services beyond S3, of course, but make sure you test the necessary scenarios thoroughly to avoid blocking legitimate requests. Not all condition keys are available in all API access scenarios and service combinations, because why would things be this easy?</p>
<h2 id="resources">Resources</h2>
<p>Here are a few other resources if you want to learn more about what the incident:</p>
<ul>
<li>
<p><a href="https://www.theregister.co.uk/2019/07/30/capital_one_hacked/">Capital One gets Capital Done: Hacker swipes personal info on 106 million US, Canadian credit card applicants</a> by The Register using a rather distasteful title but with good information. </p>
</li>
<li>
<p><a href="https://regmedia.co.uk/2019/07/29/capital_one_paige_thompson.pdf">Affidavit</a> which is the original source of what is known so far about the breach, including the alledged attacker's many OPSEC failings.</p>
</li>
<li>
<p><a href="https://securosis.com/blog/what-we-know-about-the-capital-one-data-breach">What We Know about the Capital One Data Breach</a> by <a href="https://twitter.com/rmogull">Rich Mogull</a></p>
</li>
<li>
<p><a href="https://medium.com/netflix-techblog/netflix-information-security-preventing-credential-compromise-in-aws-41b112c15179">Netflix Information Security: Preventing Credential Compromise in AWS</a> by the Netflix team and brought to my attention by <a href="https://twitter.com/0xdabbad00">Scott Piper</a></p>
</li>
</ul>Highlights from AWS re:Inforce 20192019-06-30T12:18:00-03:002019-06-30T12:18:00-03:00Alexandre Sieiratag:www.infoseczanshin.com,2019-06-30:/highlights-from-aws-reinforce-2019.html<p>A not at all impartial view of AWS' first security-only event</p><p>This week I had the pleasure of attending the first ever edition of <a href="https://reinforce.awsevents.com/">AWS re:Inforce</a>. In this post I'll try to summarize my impressions of the event, plus include comments and references to the main points of attention and associated documentation and related session videos.</p>
<h2>The Big News</h2>
<p><a href="https://aws.amazon.com/blogs/aws/new-vpc-traffic-mirroring/">VPC Traffic Mirroring</a>, was, in my opinion, the most market-relevant announcement at the conference. </p>
<p>This holds the promise of allowing several security product categories to be first-class citizens in AWS for the first time. After all, AWS has full control into what integration points they allow for traditional security vendors. They are slowly adding integration points, from early days as firewalls using Marketplace AMIs to act as NAT instances to something more explicit like <a href="https://aws.amazon.com/marketplace/solutions/security/waf-managed-rules">WAF Managed Rules</a>. Though not explicitly associated with partners, this is still a big deal for many of them.</p>
<p>Impacted product categories include Network Detection and Response, Network IDS, Network DLP and Network Forensics players. There have been a few announcements of support from vendors already, and be sure to expect more.</p>
<blockquote class="twitter-tweet" data-partner="tweetdeck"><p lang="en" dir="ltr">Oh, I was wondering what all those network traffic analysis vendors were saying there were about to announce regarding AWS ;-)</p>— Augusto Barros (@apbarros) <a href="https://twitter.com/apbarros/status/1143547645648134146?ref_src=twsrc%5Etfw">June 25, 2019</a></blockquote>
<p>As far as the customer base goes, this will mostly benefit mixed environment companies. Imagine you are a large, mature organization that has a large Bro / Zeek deployment, trained personnel and lots of bespoke analytics content built on top of it. Being able to extend it to your AWS environment and leverage your existing investments is a no-brainer. I fully expect that the main driver behind AWS developing this functionality was from large representative customers in this situation.</p>
<p>As usual this is a first release and will be surely improved upon by AWS. Major caveats I noticed so far:</p>
<ul>
<li>
<p>Despite the name, collection must be set up one <a href="https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html">ENI</a> at a time. No way to set this up across the entire VPC as you would with <a href="https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html">Flow Logs</a>.</p>
</li>
<li>
<p>The replicated traffic generated by each instance will count against the overall bandwidth available to the instance, so do your capacity planning before deploying this in production.</p>
</li>
</ul>
<h2>Other Noteworthy Announcements</h2>
<p>Out of the other many new features and announcements, these were the ones that caught my attention:</p>
<ul>
<li><a href="https://aws.amazon.com/about-aws/whats-new/2019/06/aws-security-hub-now-generally-available/">Security Hub is now GA</a>. Security Hub's main advantage is providing a single dashboard, event format and 3rd party integration point for your AWS security, including native services like <a href="https://aws.amazon.com/macie/">Macie</a> and <a href="https://aws.amazon.com/guardduty/">Guard Duty</a>. The main limitation at this point is that even though it has cross-account support, it is a per-region service. So large organizations will still have multiple "panes of glass". Hope to see AWS work on that in the future.</li>
</ul>
<iframe width="560" height="315" src="https://www.youtube.com/embed/kD6JdUAKdV8" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
<ul>
<li>
<p><a href="https://aws.amazon.com/about-aws/whats-new/2019/06/aws-control-tower-is-now-generally-available/">Control Tower is now GA</a>. Essentially AWS is taking the lessons learned from <a href="https://aws.amazon.com/solutions/aws-landing-zone/">Landing Zone</a>, which is a professional-services-only offer, and working towards allowing self-service automation of multi-account management as a native service. At this point it still is very much an MVP with several limitations, but should soon evolve into something invaluable so pay attention to future announcements from this team. The most relevant current limitations are:</p>
<ul>
<li>
<p>Will only work on new environments and has no way to incorporate an existing <a href="https://aws.amazon.com/organizations/">Organizations</a> hierarchy;</p>
</li>
<li>
<p>Does not allow you to create your own service control policies or "guard rails" as they call them in Control Tower lingo;</p>
</li>
<li>
<p>Does not allow you to customize provided guard rails except for per-account On/Off status;</p>
</li>
<li>
<p>Does not have its own API, seems at this point to be basically a UI layer on top of existing services.</p>
</li>
</ul>
</li>
</ul>
<iframe width="560" height="315" src="https://www.youtube.com/embed/2t-VkWt0rKk" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
<ul>
<li><a href="https://aws.amazon.com/blogs/aws/new-opt-in-to-default-encryption-for-new-ebs-volumes/">Opt-in to Default Encryption for EBS Volumes</a> which should make meeting compliance requirements by default a lot easier on larger organizations. Does not handle existing unencrypted EBS volumes, but then again it's simple enough to find those with <a href="https://docs.aws.amazon.com/config/latest/developerguide/encrypted-volumes.html">Config</a>. This is a per-region setting, but can be automated through API calls. Most importantly, it only seems to work with <a href="https://aws.amazon.com/blogs/aws/amazon-ec2-update-additional-instance-types-nitro-system-and-cpu-options/">Nitro</a> instances:</li>
</ul>
<blockquote class="twitter-tweet" data-lang="pt"><p lang="en" dir="ltr">The EBS default encryption feature for AWS uses Nitro, which means instance types that don't support Nitro can no longer be launched once you enable that feature. As a side-effect, I think this means this would enforce the VPC transit encryption for your EC2s.</p>— Scott Piper (@0xdabbad00) <a href="https://twitter.com/0xdabbad00/status/1145392090806755328?ref_src=twsrc%5Etfw">30 de junho de 2019</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<h2>Keynote Highlights</h2>
<p>During the <a href="https://reinforce.awsevents.com/learn/keynote/">keynote</a>, Stephen E. Schmidt (AWS VP and CISO) was the ranking executive and led the announcements. By the way don't miss out on the hilarious Corey Quinn's <a href="https://twitter.com/QuinnyPig/status/1143504072621867009">live-tweet thread</a>, and you can also take a look at non-threaded posts from <a href="https://twitter.com/fsmontenegro/status/1143508566499909637">Fernando Montenegro</a> and <a href="https://twitter.com/AlexandreSieira/status/1143514790733725700">I</a>.</p>
<p>During the keynote two moments caught my attention. The first was when Schmidt jabbed Azure and other competitors for the <a href="https://www.datacenterknowledge.com/microsoft/azure-outage-proves-hard-way-availability-zones-are-good-idea">comparatively bad track record on availability and regional redundancy</a>:</p>
<blockquote class="twitter-tweet" data-lang="pt"><p lang="en" dir="ltr">AWS throwing shade at competitors on region vs. availability zones <a href="https://twitter.com/hashtag/reInforce?src=hash&ref_src=twsrc%5Etfw">#reInforce</a> <a href="https://t.co/sg36GRjeXC">pic.twitter.com/sg36GRjeXC</a></p>— Alexandre Sieira (@AlexandreSieira) <a href="https://twitter.com/AlexandreSieira/status/1143515510589546496?ref_src=twsrc%5Etfw">25 de junho de 2019</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>The other one was having a representative from <a href="https://en.wikipedia.org/wiki/Capital_One">Capital One</a> go on stage to claim they will be entirely cloud-based by 2020. This is a major indication (if you still needed any) of the penetration that cloud adoption is having even on compliance-heavy industries. </p>
<h2>Venue and Overall Impressions</h2>
<p>The event was held at the <a href="https://www.signatureboston.com/bcec/getting-here">Boston Convention and Exhibition Center</a> on June 25<sup>th</sup> and 26<sup>th</sup>. This was my first contact with this venue, but I found it perfectly accommodated an event of this size. I did, however, find a telling lack of available lodging with some astronomical hotel rates and a severe shortage of close-by AirBnb offers even registering about one month in advance.</p>
<p>Overall, I fully support AWS' wonderful idea to start separate focused conferences. I attended <a href="https://reinvent.awsevents.com/">re:Invent</a> last year in Las Vegas and it was quite simply too big to be enjoyable. I had a much better time at <a href="https://reinforce.awsevents.com/">re:Inforce</a> and was pleasantly surprised to actually make it to a few sessions as a walk-in without needing to queue a full hour in advance.</p>
<p>The show floor had a decent size, but a portion of the size of a single of the original rooms of the RSA Conference Expo in Moscone West or East. Still, it was a single large room containing both the sponsor booths, the AWS Developer Lounge, CTF, Breakout Session spaces and tables used for lunch, a break from it all or impromptu meetings. Again a very pleasant, spacious experience.</p>
<blockquote class="twitter-tweet" data-lang="pt"><p lang="en" dir="ltr">Sponsors getting ready for <a href="https://twitter.com/hashtag/reinforce?src=hash&ref_src=twsrc%5Etfw">#reinforce</a> <a href="https://t.co/oHns6yRvm8">pic.twitter.com/oHns6yRvm8</a></p>— Andres Riancho ☁️ (@AndresRiancho) <a href="https://twitter.com/AndresRiancho/status/1143220457883230208?ref_src=twsrc%5Etfw">24 de junho de 2019</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>There were areas with supported charities, plus a couple of really nice touches I wish more conference organizers learn from. First, swag donation collectors:</p>
<blockquote class="twitter-tweet" data-lang="pt"><p lang="en" dir="ltr">This is great. Would love to see similar at all conferences. <a href="https://twitter.com/hashtag/reInforce?src=hash&ref_src=twsrc%5Etfw">#reInforce</a> <a href="https://t.co/3Y8iJmiN4T">pic.twitter.com/3Y8iJmiN4T</a></p>— Chris Eng (@chriseng) <a href="https://twitter.com/chriseng/status/1143896366592274434?ref_src=twsrc%5Etfw">26 de junho de 2019</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>Additionally, this is a good way to make the conference more inclusive:</p>
<blockquote class="twitter-tweet" data-lang="pt"><p lang="en" dir="ltr">It’s great to see conferences doing this. This is a legit huge room with chairs and lights and carpet and everything you’d expect, not some random closet. <a href="https://twitter.com/hashtag/ReInvent?src=hash&ref_src=twsrc%5Etfw">#ReInvent</a> <a href="https://t.co/5sjd96ozdl">pic.twitter.com/5sjd96ozdl</a></p>— Richard H. Boyd 🔬🎉 @ re:covery (@rchrdbyd) <a href="https://twitter.com/rchrdbyd/status/1143879178707714049?ref_src=twsrc%5Etfw">26 de junho de 2019</a></blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>In summary, my overall impression is that this was absolutely worth the time and money and I fully intend to attend <a href="https://reinforce.awsevents.com/">re:Inforce</a> again next year.</p>
<h2>Resources</h2>
<p>Here are a few other resources if you want to learn more about what went on at <a href="https://reinforce.awsevents.com/">re:Inforce</a>:</p>
<ul>
<li>
<p><a href="https://www.youtube.com/playlist?list=PLmbt_7-buHq-sx_JFNMsbkI5qHb0gWtfZ">Youtube playlist with all the recorded sessions</a> by AWS</p>
</li>
<li>
<p><a href="https://aws.amazon.com/blogs/security/reinforce-2019-wrap-up-and-session-links/">Re:Inforce 2019 wrap-up and session links</a> by AWS</p>
</li>
<li>
<p><a href="https://medium.com/@ashishrajan/aws-re-inforce-boston-2019-aws-security-conference-recap-what-they-didnt-tell-you-on-stage-9b7ecc04ddfe">AWS Re:inforce Boston 2019— AWS Security Conference: Recap (What they didn’t tell you on stage)</a> by <a href="https://medium.com/@ashishrajan">Ashish Rajan</a></p>
</li>
</ul>
<p>Thank you to <a href="https://twitter.com/dseverski">David Severski</a> for helping review an early draft of this document and providing valuable feedback.</p>Connecting AWS VPCs with StrongSwan2017-03-30T18:06:00-03:002017-03-30T18:06:00-03:00Alexandre Sieiratag:www.infoseczanshin.com,2017-03-30:/connecting-aws-vpcs-with-strongswan.html<p>Establishing a site-to-site IPsec VPN between VPCs in different regions on a budget</p><p>On my <a href="/using-openvpn-access-server-to-access-aws-vpcs.html">previous post</a>
I covered how to allow client-to-site connectivity to an AWS
<a href="https://aws.amazon.com/documentation/vpc/">VPC</a> environment.</p>
<p>On this post I decided to continue exploring AWS VPC connectivity and talk about how to
connect VPCs. If you have VPCs on the same region, you could simply use
<a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-peering.html">VPC peering</a> and
be done with it. But if your VPCs are located in different regions, you'll need to explore
your <a href="https://aws.amazon.com/answers/networking/aws-multiple-region-multi-vpc-connectivity/">options</a>.</p>
<p>I decided to test and document one of the more inexpensive and simple options I could think of,
full mesh connectivity between VPCs using IPsec site-to-site tunnels. And the <em>inexpensive</em> part
is taken care of by using <a href="http://www.stronswan.org">StrongSwan</a> 5.4.0 on
<a href="https://www.centos.org">CentOS</a> 7 to implement this.</p>
<p>Basically the scenario here is that I want to connect two VPCs on different regions:</p>
<ul>
<li>
<p>us-east-1 VPC with IP addresses in 172.16.0.0/16;</p>
</li>
<li>
<p>us-east-2 VPC with IP addresses in 172.31.0.0/16.</p>
</li>
</ul>
<p>It is a simple exercise to extrapolate this configuration to have additional VPCs connected
to these two via full mesh, so I won't get into the specifics of this here. Consider that
as homework.</p>
<h2>IP Addresses and Security Groups</h2>
<p>First, create one <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html">Elastic IP Address</a>
for each StrongSwan instance. Optionally, create a hostname for each in Route53 if you
think that will help you later on.</p>
<p>Then, create one
<a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_SecurityGroups.html">security group</a>
for each of the StrongSwan instances. Leave all outbound traffic as allowed, and create the
following inbound rules:</p>
<ul>
<li>
<p>Create an SSH rule to allow you to log into the box later on;</p>
</li>
<li>
<p>Allow <code>All traffic</code> from all of the VPC IP address ranges. In our example, this means
allowing all traffic from 172.16.0.0/16 and 172.31.0.0/16 on the security group. This is
necessary because when an instance acts a router, you can't differentiate traffic directed
to its own IP address or to one of the remote networks it can route to on the security group.
Any such differentiation will unfortunately need to be implemented internally in
<a href="https://wiki.centos.org/HowTos/Network/IPTables">iptables</a>;</p>
</li>
<li>
<p>For each of the other elastic IP addresses of StrongSwan instances it will need to
connect to, create the following rules:</p>
</li>
</ul>
<table>
<thead>
<tr>
<th>Type</th>
<th>Protocol</th>
<th>Port Range</th>
<th>Source</th>
</tr>
</thead>
<tbody>
<tr>
<td>Custom ICMP Rule - IPv4</td>
<td>Time Exceeded</td>
<td>All</td>
<td><em>elastic IP</em>/32</td>
</tr>
<tr>
<td>Custom ICMP Rule - IPv4</td>
<td>Destination Unreachable</td>
<td>All</td>
<td><em>elastic IP</em>/32</td>
</tr>
<tr>
<td>Custom ICMP Rule - IPv4</td>
<td>Echo Reply</td>
<td>N/A</td>
<td><em>elastic IP</em>/32</td>
</tr>
<tr>
<td>Custom ICMP Rule - IPv4</td>
<td>Echo Request</td>
<td>N/A</td>
<td><em>elastic IP</em>/32</td>
</tr>
<tr>
<td>Custom ICMP Rule - IPv4</td>
<td>Traceroute</td>
<td>N/A</td>
<td><em>elastic IP</em>/32</td>
</tr>
<tr>
<td>Custom Protocol</td>
<td>AH (51)</td>
<td>All</td>
<td><em>elastic IP</em>/32</td>
</tr>
<tr>
<td>Custom UDP Rule</td>
<td>UDP</td>
<td>4500</td>
<td><em>elastic IP</em>/32</td>
</tr>
<tr>
<td>Custom UDP Rule</td>
<td>UDP</td>
<td>500</td>
<td><em>elastic IP</em>/32</td>
</tr>
</tbody>
</table>
<p>The ICMP rules above serve two purposes. Firstly, the traceroute and echo reply/request ones will
make it easier for you to troubleshoot the connectivity between the StrongSwan instances. Most
importantly, though, the time exceeded and destination unreachable entries are there to allow
<a href="https://en.wikipedia.org/wiki/Path_MTU_Discovery">path MTU discovery</a> to happen properly between
StrongSwan instances communicating over the Internet.</p>
<p>Next, update all of existing security groups in each VPC to ensure these same ICMP messages are
accepted from all VPCs IP address ranges (172.16.0.0/16 and 172.31.0.0/16 in our example). The
objective here is similar: to allow troubleshooting and proper path MTU discovery to happen on
the end-to-end communications between machines on different VPCs through the VPN.</p>
<h2>Create StrongSwan Instances and Configure Linux</h2>
<p>This is what you need to keep in mind when creating the instances:</p>
<ul>
<li>
<p>Pick the instance type you'll need. Something on the <a href="https://aws.amazon.com/ec2/instance-types/#c4">c4 family</a>
for more heavy-duty traffic volumes, or something on the <a href="https://aws.amazon.com/ec2/instance-types/#t2">t2</a>
family should be more than enough for sporadic management / admin traffic;</p>
</li>
<li>
<p>Use the latest <a href="https://aws.amazon.com/marketplace/pp/B00O7WM7QW/">CentOS 7 AMI</a> to create a
new instance on a public subnet of the chosen region with the security group we recently created;</p>
</li>
<li>
<p><a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-eips.html#WorkWithEIPs">Associate</a>
the elastic IP address to the instance;</p>
</li>
<li>
<p><a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCheck">Disable the source/destination check</a>
on the instance since it will act as a router.</p>
</li>
</ul>
<p>Then, SSH into the machine (keep in mind the default username for the AMI is <code>centos</code>) so
we can configure the operating system properly. Make sure you become root for the following
configuration steps.</p>
<p>Ensure that <code>/etc/sysctl.conf</code> contains the following lines and then force them to be loaded by
running <code>sysctl -p /etc/sysctl.conf</code> or by rebooting:</p>
<div class="bash highlight"><pre><span></span><code>net.ipv4.ip_forward <span class="o">=</span> <span class="m">1</span>
net.ipv4.conf.all.send_redirects <span class="o">=</span> <span class="m">0</span>
net.ipv4.conf.default.send_redirects <span class="o">=</span> <span class="m">0</span>
net.ipv4.tcp_max_syn_backlog <span class="o">=</span> <span class="m">1280</span>
net.ipv4.icmp_echo_ignore_broadcasts <span class="o">=</span> <span class="m">1</span>
net.ipv4.conf.all.accept_source_route <span class="o">=</span> <span class="m">0</span>
net.ipv4.conf.all.accept_redirects <span class="o">=</span> <span class="m">0</span>
net.ipv4.conf.all.secure_redirects <span class="o">=</span> <span class="m">0</span>
net.ipv4.conf.all.log_martians <span class="o">=</span> <span class="m">1</span>
net.ipv4.conf.default.accept_source_route <span class="o">=</span> <span class="m">0</span>
net.ipv4.conf.default.accept_redirects <span class="o">=</span> <span class="m">0</span>
net.ipv4.conf.default.secure_redirects <span class="o">=</span> <span class="m">0</span>
net.ipv4.icmp_echo_ignore_broadcasts <span class="o">=</span> <span class="m">1</span>
net.ipv4.icmp_ignore_bogus_error_responses <span class="o">=</span> <span class="m">1</span>
net.ipv4.tcp_syncookies <span class="o">=</span> <span class="m">1</span>
net.ipv4.conf.all.rp_filter <span class="o">=</span> <span class="m">1</span>
net.ipv4.conf.default.rp_filter <span class="o">=</span> <span class="m">1</span>
net.ipv4.tcp_mtu_probing <span class="o">=</span> <span class="m">1</span>
</code></pre></div>
<p>As a side note, it is strongly recommended that you include <code>net.ipv4.tcp_mtu_probing = 1</code> on the
<code>sysctl.conf</code> of all of your Linux EC2 instances, since they use jumbo frames by default.</p>
<p>Let's make sure the machine is fully patched, that we can use EPEL and that we install StrongSwan
by issuing the following commands:</p>
<div class="bash highlight"><pre><span></span><code>yum install epel-release
yum repolist
yum update
yum install strongswan
systemctl <span class="nb">enable</span> strongswan
</code></pre></div>
<p>In order to ensure the cryptography and logging work properly, the system needs to have proper time
synchronization. Make sure <a href="https://en.wikipedia.org/wiki/Network_Time_Protocol">NTP</a> is installed and
configured to run on system start:</p>
<div class="bash highlight"><pre><span></span><code>yum install ntp
systemctl <span class="nb">enable</span> ntpd
</code></pre></div>
<p>Replace the <code>server</code> configuration entries in <code>/etc/ntp.conf</code> so the AWS recommended NTP
server pool is used:</p>
<div class="bash highlight"><pre><span></span><code>server <span class="m">0</span>.amazon.pool.ntp.org iburst
server <span class="m">1</span>.amazon.pool.ntp.org iburst
server <span class="m">2</span>.amazon.pool.ntp.org iburst
server <span class="m">3</span>.amazon.pool.ntp.org iburst
</code></pre></div>
<p>Finally, restart the NTP service with <code>systemctl restart ntpd</code> and check that it is working properly
with <code>ntpq -p</code>.</p>
<h2>Configuring StrongSwan</h2>
<p>We'll configure StrongSwan to use RSA keys for authentication, so the first step is to create
those keys and associate them with the servers in the StrongSwan configuration.</p>
<p>On each StrongSwan instance, create its own RSA key. This is how you would do it on the us-east-1
StrongSwan instance:</p>
<div class="bash highlight"><pre><span></span><code><span class="nb">cd</span> /etc/strongswan/ipsec.d/private/
openssl genrsa -out us-east-1.key <span class="m">4096</span>
chmod og-r us-east-1.key
openssl rsa -in us-east-1.key -pubout > ../certs/us-east-1.pub
</code></pre></div>
<p>Once you do that, you need to edit <code>/etc/strongswan/ipsec.secrets</code> to let StrongSwan know what to
do with the private key. Add a line to that file that associates each instance's own elastic IP address
to the key file. Assuming the elastic IP address of the us-east-1 StrongSwan instance is <code>1.2.3.4</code>,
this is what that line would look like:</p>
<div class="bash highlight"><pre><span></span><code><span class="m">1</span>.2.3.4 : RSA us-east-1.key
</code></pre></div>
<p>Then, you copy each StrongSwan instance's <code>.pub</code> file to the <code>/etc/strongswan/ipsec.d/certs</code>
directory of each of the other StrongSwan instances. In our example, if you were on the us-east-1
instance you would see something like this:</p>
<div class="bash highlight"><pre><span></span><code>$ find /etc/strongswan/ipsec.d/ -name *.key
/etc/strongswan/ipsec.d/private/us-east-1.key
$ find /etc/strongswan/ipsec.d/ -name *.pub
/etc/strongswan/ipsec.d/certs/us-east-1.pub
/etc/strongswan/ipsec.d/certs/us-east-2.pub
</code></pre></div>
<p>Finally, you configure <code>/etc/strongswan/ipsec.conf</code> to tie it all together. This is what the
configuration file would look like the the elastic IP for the us-east-1 and us-east-2 instances
were 1.2.3.4 and 2.3.4.5, respectively:</p>
<div class="bash highlight"><pre><span></span><code>config setup
<span class="c1"># strictcrlpolicy=yes</span>
<span class="c1"># uniqueids = no</span>
conn %default
<span class="nv">fragmentation</span><span class="o">=</span>force
<span class="nv">dpdaction</span><span class="o">=</span>restart
<span class="nv">ike</span><span class="o">=</span>aes192gcm16-aes128gcm16-aes192-prfsha256-ecp256-ecp521,aes192-sha256-modp3072
<span class="nv">esp</span><span class="o">=</span>aes192gcm16-aes128gcm16-aes192-ecp256,aes192-sha256-modp3072#
<span class="nv">keyingtries</span><span class="o">=</span>%forever
<span class="nv">keyexchange</span><span class="o">=</span>ikev2
<span class="nv">authby</span><span class="o">=</span>rsasig
<span class="nv">forceencaps</span><span class="o">=</span>yes
<span class="nv">leftid</span><span class="o">=</span><span class="m">1</span>.2.3.4
<span class="nv">leftrsasigkey</span><span class="o">=</span>us-east-1.pub
<span class="nv">leftsubnet</span><span class="o">=</span><span class="m">172</span>.16.0.0/16
<span class="c1"># Add connections here.</span>
conn us-east-2
<span class="nv">right</span><span class="o">=</span><span class="m">2</span>.3.4.5
<span class="nv">rightsubnet</span><span class="o">=</span><span class="m">172</span>.31.0.0/16
<span class="nv">rightrsasigkey</span><span class="o">=</span>us-east-2.pub
<span class="nv">auto</span><span class="o">=</span>start
</code></pre></div>
<p>Keep in mind that <code>left</code> in StrongSwan parlance means the side of the VPN that is local to
the instance you are configuring, and <code>right</code> is the remote side. So the configuration file on
the us-east-2 instance would look like this:</p>
<div class="bash highlight"><pre><span></span><code>config setup
<span class="c1"># strictcrlpolicy=yes</span>
<span class="c1"># uniqueids = no</span>
conn %default
<span class="nv">fragmentation</span><span class="o">=</span>yes
<span class="nv">dpdaction</span><span class="o">=</span>restart
<span class="nv">ike</span><span class="o">=</span>aes192gcm16-aes128gcm16-aes192-prfsha256-ecp256-ecp521,aes192-sha256-modp3072
<span class="nv">esp</span><span class="o">=</span>aes192gcm16-aes128gcm16-aes192-ecp256,aes192-sha256-modp3072#
<span class="nv">keyingtries</span><span class="o">=</span>%forever
<span class="nv">keyexchange</span><span class="o">=</span>ikev2
<span class="nv">authby</span><span class="o">=</span>rsasig
<span class="nv">forceencaps</span><span class="o">=</span>yes
<span class="nv">leftid</span><span class="o">=</span><span class="m">2</span>.3.4.5
<span class="nv">leftrsasigkey</span><span class="o">=</span>us-east-2.pub
<span class="nv">leftsubnet</span><span class="o">=</span><span class="m">172</span>.31.0.0/16
<span class="c1"># Add connections here.</span>
conn us-east-1
<span class="nv">right</span><span class="o">=</span><span class="m">1</span>.2.3.4
<span class="nv">rightsubnet</span><span class="o">=</span><span class="m">172</span>.16.0.0/16
<span class="nv">rightrsasigkey</span><span class="o">=</span>us-east-1.pub
<span class="nv">auto</span><span class="o">=</span>start
</code></pre></div>
<p>Please review the StrongSwan
<a href="https://wiki.strongswan.org/projects/strongswan/wiki/IpsecConf">documentation on ipsec.conf</a> to
better understand some of the choices I've made there, and tweak the setup to meet your needs.
I wouldn't change the configuration on the <code>fragmentation</code> and <code>forceencaps</code> options, though,
since I had problems if they were not set as above.</p>
<p>Once you've set all of this up, run <code>systemctl restart strongswan</code> and monitor the logs with
<code>tail -f /var/log/messages | grep charon</code> for log entries related to the IPsec tunnel
negotiations and authentication.</p>
<p>Hopefully by now you will be able to ping us-east-2's StrongSwan instance internal (172.31.0.x)
IP address from the SSH session on us-east-1's StrongSwan instance.</p>
<h2>Routing</h2>
<p>Finally, in order to allow machines on one region to talk to machines and services on the other,
we'll need to update the
<a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Route_Tables.html">route tables</a>.</p>
<p>What you need to do is to
<a href="http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Route_Tables.html#AddRemoveRoutes">add a new route</a>
that tells machines on a region that in order to talk to the addresses on the other regions, they
must go through the StrongSwan instance. </p>
<p>So in our example, you should add a new route to all routing tables in us-east-1 that has a <code>Destination</code>
of 172.31.0.0/16, and a <code>Target</code> that is the instance ID of the us-east-1 StrongSwan instance.</p>
<p>Conversely, you should add a new route to all routing tables in us-east-2 that has a <code>Destination</code> of
172.16.0.0/16, and a <code>Target</code> that is the instance ID of the us-east-2 StrongSwan instance.</p>
<p>Finally, make sure that the security groups of services that need to be accessed across the VPN
will now allow the IP addresses of the remote machines in. Once you do that, you can then test the
communication between regions successfully. Of course, if you enabled ICMP as recommended above,
you should be able to ping any instance in us-east-2 from any instance in us-east-1 and vice-versa
by now.</p>
<h2>Availability Concerns</h2>
<p>You could achieve some level of redundancy and distribution of load by increasing the number
of VPN concentrator instances you stand up. </p>
<p>One idea would be to create one VPN concentrator per availability zone instead of just one per
region. In this scenario even if one availability zone (or its StrongSwan instance) become
unavailable, the rest of the availability zones will remain connected.</p>
<p>This is a high level guide of what that would entail in addition to what was discussed
above:</p>
<ul>
<li>Create the additional StrongSwan instances as per the instructions above;</li>
<li>Separate the routing tables per availability zone and assign each one to its corresponding
subnets;</li>
<li>Update <code>ipsec.conf</code> on all machines to have one connection for each VPN concentrator. Also
update each one's <code>leftsubnet</code> and <code>rightsubnet</code> definitions so that each server is only
responsible for the IP address ranges of the subnets in its availability zone.</li>
</ul>
<p>I have not covered implementing HA on StrongSwan, though apparently that is
<a href="https://wiki.strongswan.org/projects/strongswan/wiki/HighAvailability">supported</a> as well.
If you get this working let me know.</p>
<h2>Additional Recommendations</h2>
<p>A few security-minded tips that I would recommend you implement:</p>
<ul>
<li>
<p>Ensure you close off SSH access to the StrongSwan instances after you're done configuring them,
by removing the applicable Security Group inbound rule. You can always allow it temporarily again
on the Security Group if and when you need it.</p>
</li>
<li>
<p>Install the
<a href="http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/QuickStartEC2Instance.html">CloudWatch Logs Agent</a>
on the machine, remember we covered this already
<a href="/using-openvpn-access-server-to-access-aws-vpcs.html">here</a>. Make sure you collect at least the
following files: <code>/var/log/messages</code>, <code>/var/log/secure</code> and <code>/var/log/audit/audit.log</code>. </p>
</li>
<li>
<p>Harden the operating system and make sure to keep install security updates as they become available.</p>
</li>
</ul>Using OpenMP with R packages in OS X2016-07-21T18:01:00-03:002016-07-21T18:01:00-03:00Alexandre Sieiratag:www.infoseczanshin.com,2016-07-21:/using-openmp-with-r-packages-in-os-x.html<p>Using OpenMP to allow parallel computation in R packages on Mac OS X</p><p>Lots of R packages have underlying implementations in C and C++, so they need to be compiled to
function properly. But when you install a package in OS X you will probably get a pre-compiled version
of said package from <a href="https://cran.r-project.org/">CRAN</a> that used their build environment and the
default package options.</p>
<p>Using a pre-compiled package is almost never a problem, except when the default options don't fit
your requirements. In that case, you'll need to install them from source and make sure you get
the compilation environment right.</p>
<p>I recently ran into a package that had the capability of using <a href="http://openmp.org/">OpenMP</a> to perform
computation in parallel and take advantage of my MacBook Pro's multi-core processor to the fullest.
If you are interested in how this works and how to integrate it into your own R packages, check out
the <a href="https://wrathematics.github.io/RparallelGuide/">Parallelism, R, and OpenMP</a> guide by
<a href="https://wrathematics.github.io/">Drew Schmidt</a>.</p>
<p>So this is all awesome, but there's a catch. There's always a catch, right?</p>
<p>Turns out the binary version of packages available at <a href="https://cran.r-project.org/">CRAN</a> are not
compiled to use OpenMP, since not all systems are equipped to support it. Moreover, if you try to
install the package from source on OS X, it will try to use <code>clang</code> / <code>clang++</code> which in my experience
does not support OpenMP either.</p>
<p>Before I get into how I was eventually able to fix this, if you don't have <a href="http://brew.sh/">homebrew</a>
installed on your Mac yet, stop everything and go install it. Yes, now. I'll wait.</p>
<p>So, I found lots of outdated references to a homebrew package called <code>clang-omp</code>, but as you can see
at its <a href="https://clang-omp.github.io/">website</a> the project was moved elsewhere. And it doesn't exist
in homebrew anymore, so that is a dead end.</p>
<p>What finally worked for me was to install GCC 6 using homebrew, and using it to compile the R packages
with OpenMP support. This is how you do it using homebrew:</p>
<div class="sh highlight"><pre><span></span><code>brew install gcc --without-multilib
</code></pre></div>
<p>It's important to include <code>--without-multilib</code> since it apparently
<a href="http://stackoverflow.com/questions/30049486/what-does-gcc-without-multilib-mean">creates problems</a> with
OpenMP. Unless you're cross-compiling to different architectures you probably won't miss it anyway.</p>
<p>If you already had gcc installed with homebrew, just replace <code>install</code> above with <code>reinstall</code>.</p>
<p>After you do that, you'll notice that gcc executables should have been linked to <code>/usr/local/bin</code>. Next,
what you'll need to do is to create (or edit) <code>~/.R/Makevars</code> to reference them:</p>
<div class="make highlight"><pre><span></span><code><span class="nv">CC</span><span class="o">=</span>/usr/local/bin/gcc-6
<span class="nv">CXX</span><span class="o">=</span>/usr/local/bin/g++-6
<span class="nv">CXX1X</span><span class="o">=</span>/usr/local/bin/g++-6
<span class="nv">CXX11</span><span class="o">=</span>/usr/local/bin/g++-6
<span class="nv">SHLIB_CXXLD</span><span class="o">=</span>/usr/local/bin/g++-6
<span class="nv">FC</span><span class="o">=</span>/usr/local/bin/gfortran-6
<span class="nv">F77</span><span class="o">=</span>/usr/local/bin/gfortran-6
<span class="nv">MAKE</span><span class="o">=</span>make -j8
<span class="nv">SHLIB_OPENMP_CFLAGS</span><span class="o">=</span>-fopenmp
<span class="nv">SHLIB_OPENMP_CXXFLAGS</span><span class="o">=</span>-fopenmp
<span class="nv">SHLIB_OPENMP_FCFLAGS</span><span class="o">=</span>-fopenmp
<span class="nv">SHLIB_OPENMP_FFLAGS</span><span class="o">=</span>-fopenmp
</code></pre></div>
<p>Make sure you adjust the major version on the executable file names to match what homebrew installed in
the last step.</p>
<p>After all this, uninstall the R packages you want to with OpenMP with, and
<a href="http://stackoverflow.com/questions/1474081/how-do-i-install-an-r-package-from-source">reinstall them from source</a>.</p>
<p>That's it! Hopefully you will now have a working OpenMP-enabled R package to work with.</p>Introducing SnakeCharmR2016-06-13T19:17:00-03:002016-06-13T19:17:00-03:00Alexandre Sieiratag:www.infoseczanshin.com,2016-06-13:/introducing-snakecharmr.html<p>A modern R package to allow Python to be called from R</p><p>I have been using <a href="https://cran.r-project.org/web/packages/rPython/index.html">rPython</a> for some time
in Linux and Mac OS X environments. When I first found out about it, the idea of using JSON as a way to
serialize and deserialize values between Python and R was beautifully simple and intuitive.</p>
<p>However, over time I realized <a href="https://cran.r-project.org/web/packages/rPython/index.html">rPython</a>
suffered from a more than a few issues. Plus, I noticed that the codebase hadn't been updated in a long
time, was based on direct C code and used RJSONIO for the underlying JSON conversions.</p>
<p>So I decided to create a new package called <a href="https://github.com/asieira/SnakeCharmR">SnakeCharmR</a> inspired
by <a href="https://cran.r-project.org/web/packages/rPython/index.html">rPython</a> but with the following differences:</p>
<ul>
<li>
<p>Use <a href="https://cran.r-project.org/web/packages/Rcpp/index.html">Rcpp</a> instead of direct R to C bindings;</p>
</li>
<li>
<p>Replace <a href="https://cran.r-project.org/web/packages/RJSONIO/index.html">RJSONIO</a> with the much more
well-behaved <a href="https://cran.r-project.org/web/packages/jsonlite/index.html">jsonlite</a> for JSON serialization
/ deserialization;</p>
</li>
<li>
<p>Allow the calling code to have full control of the options to be passed to <code>jsonlite::fromJSON</code> and
<code>jsonlite::toJSON</code>;</p>
</li>
<li>
<p>Use <a href="https://cran.r-project.org/web/packages/testthat/index.html">testthat</a> for unit testing as
much as possible, including <a href="https://travis-ci.org/asieira/SnakeCharmR">Travis</a> and
<a href="https://codecov.io/gh/asieira/SnakeCharmR">Codecov</a> for automation of tests and coverage.</p>
</li>
</ul>
<p>The package is not a drop-in replacement to rPython, but most of existing code will require very few updates
to work with it. Most of them will be related to the differences between RJSONIO and jsonlite.</p>
<h2>Installation</h2>
<p>The package has not yet been submitted to CRAN, so currently this is how you install it:</p>
<div class="r highlight"><pre><span></span><code><span class="o">></span> <span class="nf">library</span><span class="p">(</span><span class="n">devtools</span><span class="p">)</span>
<span class="o">></span> <span class="nf">install_github</span><span class="p">(</span><span class="s">"asieira/SnakeCharmR"</span><span class="p">)</span>
</code></pre></div>
<p>Please refer to the <a href="https://github.com/asieira/SnakeCharmR/blob/master/README.md">README</a> for more details
on linking with the proper Python version.</p>
<h2>Comparison with <a href="https://cran.r-project.org/web/packages/rPython/index.html">rPython</a></h2>
<p>A few examples of differences in the way <a href="https://github.com/asieira/SnakeCharmR">SnakeCharmR</a> and
<a href="https://cran.r-project.org/web/packages/rPython/index.html">rPython</a> handle some situations:</p>
<ul>
<li>
<p><a href="https://github.com/asieira/SnakeCharmR">SnakeCharmR</a> will correctly remove any temporary values used
for function/method arguments and/or serialized values from the Python environment. In the case
of <a href="https://cran.r-project.org/web/packages/rPython/index.html">rPython</a>, they would remain in memory
until overwritten by another operation.</p>
</li>
<li>
<p>Handling of strings with single quotes:</p>
</li>
</ul>
<div class="r highlight"><pre><span></span><code><span class="o">></span> <span class="nf">library</span><span class="p">(</span><span class="n">rPython</span><span class="p">)</span>
<span class="o">></span> <span class="nf">python.assign</span><span class="p">(</span><span class="s">"a"</span><span class="p">,</span> <span class="s">"'"</span><span class="p">)</span>
<span class="n">File</span> <span class="s">"<string>"</span><span class="p">,</span> <span class="n">line</span> <span class="m">2</span>
<span class="n">a</span> <span class="o">=</span><span class="s">' [ "'"</span><span class="err"> ] '</span>
<span class="o">^</span>
<span class="n">SyntaxError</span><span class="o">:</span> <span class="n">EOL</span> <span class="n">while</span> <span class="n">scanning</span> <span class="n">string</span> <span class="n">literal</span>
</code></pre></div>
<div class="r highlight"><pre><span></span><code><span class="o">></span> <span class="nf">library</span><span class="p">(</span><span class="n">SnakeCharmR</span><span class="p">)</span>
<span class="n">SnakeCharmR</span> <span class="n">v1.0.2</span> <span class="o">-</span> <span class="n">R</span> <span class="n">and</span> <span class="n">Python</span> <span class="n">Integration</span>
<span class="n">Contribute</span> <span class="n">and</span> <span class="n">submit</span> <span class="n">issues</span> <span class="n">at</span> <span class="n">https</span><span class="o">://</span><span class="n">github.com</span><span class="o">/</span><span class="n">asieira</span><span class="o">/</span><span class="n">SnakeCharmR</span>
<span class="o">></span> <span class="nf">py.assign</span><span class="p">(</span><span class="s">"a"</span><span class="p">,</span> <span class="s">"'"</span><span class="p">)</span>
<span class="o">></span> <span class="nf">py.get</span><span class="p">(</span><span class="s">"a"</span><span class="p">)</span>
<span class="p">[</span><span class="m">1</span><span class="p">]</span> <span class="s">"'"</span>
</code></pre></div>
<ul>
<li>Handling of strings with unicode values:</li>
</ul>
<div class="r highlight"><pre><span></span><code><span class="o">></span> <span class="nf">library</span><span class="p">(</span><span class="n">rPython</span><span class="p">)</span>
<span class="o">></span> <span class="nf">python.assign</span><span class="p">(</span><span class="s">"a"</span><span class="p">,</span> <span class="s">"áéíóú"</span><span class="p">)</span>
<span class="o">></span> <span class="nf">python.get</span><span class="p">(</span><span class="s">"a"</span><span class="p">)</span>
<span class="p">[</span><span class="m">1</span><span class="p">]</span> <span class="s">"\xe1\xe9\xed\xf3\xfa"</span>
<span class="o">></span> <span class="nf">Encoding</span><span class="p">(</span><span class="nf">python.get</span><span class="p">(</span><span class="s">"a"</span><span class="p">))</span>
<span class="p">[</span><span class="m">1</span><span class="p">]</span> <span class="s">"unknown"</span>
</code></pre></div>
<div class="r highlight"><pre><span></span><code><span class="o">></span> <span class="nf">library</span><span class="p">(</span><span class="n">SnakeCharmR</span><span class="p">)</span>
<span class="n">SnakeCharmR</span> <span class="n">v1.0.2</span> <span class="o">-</span> <span class="n">R</span> <span class="n">and</span> <span class="n">Python</span> <span class="n">Integration</span>
<span class="n">Contribute</span> <span class="n">and</span> <span class="n">submit</span> <span class="n">issues</span> <span class="n">at</span> <span class="n">https</span><span class="o">://</span><span class="n">github.com</span><span class="o">/</span><span class="n">asieira</span><span class="o">/</span><span class="n">SnakeCharmR</span>
<span class="o">></span> <span class="nf">py.assign</span><span class="p">(</span><span class="s">"a"</span><span class="p">,</span> <span class="s">"áéíóú"</span><span class="p">)</span>
<span class="o">></span> <span class="nf">py.get</span><span class="p">(</span><span class="s">"a"</span><span class="p">)</span>
<span class="p">[</span><span class="m">1</span><span class="p">]</span> <span class="s">"áéíóú"</span>
<span class="o">></span> <span class="nf">Encoding</span><span class="p">(</span><span class="nf">py.get</span><span class="p">(</span><span class="s">"a"</span><span class="p">))</span>
<span class="p">[</span><span class="m">1</span><span class="p">]</span> <span class="s">"UTF-8"</span>
</code></pre></div>
<ul>
<li>Improved exception handling, which in <a href="https://github.com/asieira/SnakeCharmR">SnakeCharmR</a> now
includes a full stack trace to help debugging errors that happen in Python:</li>
</ul>
<div class="r highlight"><pre><span></span><code><span class="o">></span> <span class="nf">library</span><span class="p">(</span><span class="n">rPython</span><span class="p">)</span>
<span class="o">></span> <span class="nf">python.exec</span><span class="p">(</span><span class="s">"def a():\n return b()"</span><span class="p">)</span>
<span class="o">></span> <span class="nf">python.exec</span><span class="p">(</span><span class="s">"def b():\n raise Exception('uh oh')"</span><span class="p">)</span>
<span class="o">></span> <span class="nf">python.call</span><span class="p">(</span><span class="s">"a"</span><span class="p">)</span>
<span class="n">Error</span> <span class="n">in</span> <span class="nf">python.exec</span><span class="p">(</span><span class="n">python.command</span><span class="p">)</span> <span class="o">:</span> <span class="n">uh</span> <span class="n">oh</span>
</code></pre></div>
<div class="r highlight"><pre><span></span><code><span class="o">></span> <span class="nf">library</span><span class="p">(</span><span class="n">SnakeCharmR</span><span class="p">)</span>
<span class="n">SnakeCharmR</span> <span class="n">v1.0.2</span> <span class="o">-</span> <span class="n">R</span> <span class="n">and</span> <span class="n">Python</span> <span class="n">Integration</span>
<span class="n">Contribute</span> <span class="n">and</span> <span class="n">submit</span> <span class="n">issues</span> <span class="n">at</span> <span class="n">https</span><span class="o">://</span><span class="n">github.com</span><span class="o">/</span><span class="n">asieira</span><span class="o">/</span><span class="n">SnakeCharmR</span>
<span class="o">></span> <span class="nf">py.exec</span><span class="p">(</span><span class="s">"def a():\n return b()"</span><span class="p">)</span>
<span class="o">></span> <span class="nf">py.exec</span><span class="p">(</span><span class="s">"def b():\n raise Exception('uh oh')"</span><span class="p">)</span>
<span class="o">></span> <span class="nf">py.call</span><span class="p">(</span><span class="s">"a"</span><span class="p">)</span>
<span class="n">Error</span> <span class="n">in</span> <span class="nf">py.exec</span><span class="p">(</span><span class="nf">sprintf</span><span class="p">(</span><span class="s">"_SnakeCharmR_return = json.dumps(%s(%s))"</span><span class="p">,</span> <span class="n">fname</span><span class="p">,</span> <span class="o">:</span>
<span class="nf">Traceback </span><span class="p">(</span><span class="n">most</span> <span class="n">recent</span> <span class="n">call</span> <span class="n">last</span><span class="p">)</span><span class="o">:</span>
<span class="n">File</span> <span class="s">"<string>"</span><span class="p">,</span> <span class="n">line</span> <span class="m">2</span><span class="p">,</span> <span class="n">in</span> <span class="o"><</span><span class="n">module</span><span class="o">></span>
<span class="n">File</span> <span class="s">"<string>"</span><span class="p">,</span> <span class="n">line</span> <span class="m">3</span><span class="p">,</span> <span class="n">in</span> <span class="n">a</span>
<span class="n">File</span> <span class="s">"<string>"</span><span class="p">,</span> <span class="n">line</span> <span class="m">3</span><span class="p">,</span> <span class="n">in</span> <span class="n">b</span>
<span class="n">Exception</span><span class="o">:</span> <span class="n">uh</span> <span class="n">oh</span>
</code></pre></div>
<h2>Future Improvements</h2>
<p>There are still a few pending issues to work on before SnakeCharmR can be submitted to CRAN, check
out <a href="https://github.com/asieira/SnakeCharmR/issues">GitHub issues</a> and help if you can.</p>
<p>The most relevant gap is testing and ensuring SnakeCharmR works correctly on Windows. If any of you
have had experience with compiling Rcpp packages that use external libraries on Windows, your help
will be much appreciated.</p>
<p>Many thanks to <a href="https://twitter.com/hrbrmstr">Bob Rudis</a> for his invaluable help in getting this
project going. </p>In Memoriam: Lucas Eustaquio Gomes da Silva (August 27th 1982 - April 24th, 2016)2016-04-24T19:22:00-03:002016-04-24T19:22:00-03:00Alexandre Sieiratag:www.infoseczanshin.com,2016-04-24:/in-memoriam-lucas-eustaquio-gomes-da-silva-august-27th-1982-april-24th-2016.html<p>Celebrating the life and achievements of an outstanding man.</p><p>Today myself and the rest of the Niddel team were devastated to learn that our dear friend
<a href="https://br.linkedin.com/in/lucas-eustaquio-gomes-da-silva-16522816">Lucas Eustaquio Gomes da Silva</a>,
our Senior Data Scientist since August 2013, has passed away after a brave battle against an aggressive form of
<a href="http://kickingcancerass.blogspot.com.br/">cancer</a>. </p>
<p>Our deepest condolences to his family, which are in our thoughts in this moment of pain and mourning.
The funeral service will be held in <a href="http://www.bosqueesperanca.com.br/">Belo Horizonte</a> tomorrow, April 25th,
at 10am BRT.</p>
<p>Lucas was a fun, brilliant, confident and hard-working man. We are thankful for the time we had to
work with and learn from him, and are truly saddened by how tragically short that time was. Today the
world loses an amazing human being, and one of its
<a href="http://blog.kaggle.com/2016/02/22/profiling-top-kagglers-leustagos-current-7-highest-1/">most talented data scientists</a>,
which was respected and cherished by his
<a href="https://www.kaggle.com/forums/f/15/kaggle-forum/t/20388/data-scientist-hero">colleagues and peers</a>.</p>
<p>Rest in peace, Lucas. You will be missed. We will do our best to ensure your work lives on - that the dream
you shared with us will be realized, whatever the odds.</p>Threat Intelligence Indicators are not Signatures2016-02-13T02:03:00-02:002016-02-13T02:03:00-02:00Alexandre Sieiratag:www.infoseczanshin.com,2016-02-13:/threat-intelligence-indicators-are-not-signatures.html<p>A common misunderstanding that can lead to a lot of pain.</p><p>I have recently participated in a
<a href="https://www.blackhat.com/html/webcast/01212016-data-driven-threat-intelligence.html">Black Hat webcast</a>
with <a href="https://www.twitter.com/bhaskar_vk">Bhaskar Karambelkar</a>, which was sponsored by
<a href="https://threatconnect.com/">ThreatConnect</a>. This was related to the Black Hat 2015 session called
<a href="https://youtu.be/6JMEKnes-w0?list=PLnKLzDQgx6bPRfNuf3sA2Sy2JDjbcg0P5">Data-Driven Threat Intelligence: Metrics On Indicator Dissemination And Sharing</a>,
which I had the pleasure to co-present with my good friend <a href="https://twitter.com/alexcpsec">Alex Pinto</a>.</p>
<p>At the end of the webcast, someone asked me a question about a comment I had made on how threat intelligence indicators
have multiple uses, but should not be used as signatures. One of the audience members was a bit baffled by this, and I
am sure he is not alone. </p>
<p>So let's focus on automating the use of simple network indicators (IP addresses, domain names and URLs, mostly) that
most companies will obtain from public or private threat intelligence feeds. Let me show why <em>using them directly as
signatures</em>, such as automatically generating IDS signatures and/or SIEM rules to alert or block on direct matches to
them, is very troublesome. Organizations that do that will, in my experience, be most likely flooded with false
positives.</p>
<p><img alt="False positives, false positives everywhere!" src="https://www.infoseczanshin.com/images/indicators-are-not-signatures/false-positives.png"></p>
<p>Let me go over a (far from complete) list of reasons why.</p>
<h2>Affirming the Consequent Fallacy</h2>
<p>In order to reliably generate an alert based on network traffic, you need to identify a situation in which the
probability of that traffic being malicious is reasonably high. How high it needs to be depends on your organization's
tolerance to false positives. So you need to satisfy a requirement <code>P(malicious | traffic) > threshold</code>.</p>
<p>Threat intelligence network indicators are data points that say <em>we observed a threat actor, malware or tool A generate
traffic with the characteristics M, N and O and to the external locations X, Y and Z</em>. Notice how <strong>that alone does not
equal any of the following claims</strong>:</p>
<ol>
<li>
<p>Most or all of the traffic with characteristics M, N or O to destinations X, Y, Z are caused by A;</p>
</li>
<li>
<p>A always causes traffic with characteristics M, N or O to destinations X, Y, Z.</p>
</li>
</ol>
<p>Do you notice the mismatch? Most people will erroneously equate the claim <em>attackers do X</em> to <em>I can safely alert when
X occurs in my environment</em>. It's related to the
<a href="https://en.wikipedia.org/wiki/Affirming_the_consequent">affirming the consequent fallacy</a>,
but it's even more striking because the feeds are not even making claim 2 above, which would be required for the classic
form of the fallacy.</p>
<p>To give you an example, you could find a perfectly valid indicator saying a piece of malware uses something like
a public API from Google, Dropbox or anyone else just to verify whether it can connect to the Internet. Or it could
use some publicly available service to identify which public IP address it is reaching the Internet from, and its
geolocation, in particular in the case of targeted attacks.</p>
<p>This sort of indicator can still be really useful if you are doing DFIR or hunting, as it allows you to narrow down
compromised machines on the network, or let you know which forensic data to investigate first. But it should be obvious
by now that generating an IDS or SIEM alert for every machine on your network that behaves in a similar manner would be
a really bad idea.</p>
<h2>What should I match it against?</h2>
<p>When you get an intelligence feed, it might contain indicators that are indicative of several different kinds of
malicious behaviors. In particular the paid feeds will contain a mix of human-readable context in the form a report,
and also the machine-readable indicators associated with each report. </p>
<p>The problem is that it can be very hard to automatically determine in which context each technical indicator can be
applied. In the case of IP addresses, for example, very rarely does the machine-readable data allow you to unambigously
determine something as simple as whether it is associated with <em>inbound</em> traffic, <em>outbound</em> traffic or both.</p>
<p>In case you are not familiar with the terminology, the definition of <em>inbound</em> and <em>outbound</em> I'm referring to is the
one used in <a href="https://github.com/mlsecproject/combine">combine</a> and <a href="https://github.com/mlsecproject/tiq-test">tiq-test</a>.
Keeping your organization as the point of reference, <em>inbound</em> indicators would refer to traffic originating from the
open Internet towards your organization's public assets: port scanning, credential brute forcing, automated or manual
exploitation of Internet-facing services, etc. <em>Outbound</em> indicators, on the other hand, would be associated with
traffic originating from inside your organization's network towards the open Internet, such as data exfiltration, C&C
traffic, downloading of malware or client-side exploits.</p>
<p>So knowing which traffic <em>direction</em> each indicator applies to would be the most basic way to reduce false positives,
and it is often not available for use on an automated fashion.</p>
<h2>"Helpful" Feed Providers</h2>
<p>Imagine an analyst reverses a new malware or RAT sample and identifies that it uses a particular URL to talk back to
its creator. He will of course generate an indicator in the report for that URL. However, sometimes the feed provider
will go one step further and think "but what about people that want to match this in netflow, firewall or DNS logs?",
and do you the favor of also generating indicators for:</p>
<ol>
<li>
<p>The hostname of the URL, so you can match this on DNS;</p>
</li>
<li>
<p>The IP addresses that hostname resolved to at the time of the analysis.</p>
</li>
</ol>
<p>This creates all sort of problems and piles onto the false positives.</p>
<p>First, extracting the hostname is not always appropriate. Attackers might control the entire content served under that
hostname (think DGAs, a single compromised web server that attacker fully controls). However, it might also be a portal
with completely independent sub-sites hosted under different paths that share no infrastructure except for a load
balancer that routes requests appropriately. Or it might be something like a Google Drive or Dropbox link, or a URL
shortener. So considering the entire domain to be compromised / malicious because of a few URLs within it is often
a step too far.</p>
<p>It's even worse when resolving domain names to IP addresses, even for domains completely dedicated to malicious
purposes. Firstly, we know that miscreants can and will
<a href="https://en.wikipedia.org/wiki/Fast_flux">switch the IP addresses a domain resolves to often</a>, and the IPs you
receive will most likely be outdated by the time you get to use them. Second, it's not uncommon for malicious domains
to be using a service like <a href="https://www.cloudflare.com/">CloudFlare</a>, <a href="https://www.incapsula.com/">Incapsula</a> or a
shared hosting service. So if you resolve the domain, you'll get an IP address that's shared with possibly hundreds of
benign websites. Or it could be temporarily parked at a benign IP address such as 8.8.8.8. Again, knowing that a domain
is malicious does not necessarily mean that all of the IPs it resolves to are mostly or completely malicious as well.</p>
<p>Feed providers, take note: having this extra information would be more helpful if it was possible to distinguish the
<em>principled</em> indicators from the <em>derived</em> ones. But alas, this information is often not present in the
computer-readable indicators.</p>
<h2>Conclusion</h2>
<p>I hope this article helps people realize that organizations need to put proper processes and/or automation in place to
overcome the problems identified above if they decide to use threat intelligence indicators for detection. Even though
threat intelligence indicators are really valuable allies to information security monitoring and DFIR initiatives, they
are not signatures and should be used with appropriate care.</p>Test Data to Develop Splunk Content2016-02-02T17:40:00-02:002016-02-02T17:40:00-02:00Alexandre Sieiratag:www.infoseczanshin.com,2016-02-02:/test-data-to-develop-splunk-content.html<p>Using Splunk eventgen to create sample events from different devices.</p><p><a href="http://splunk.com">Splunk</a> is the SIEM platform I have been working the most with recently. It is a joy to work with
because it has embraced the development paradigm, and organizes standard and user-developed content into apps. And it
lets you write code in proper languages like Python or Java to process the data in ways that go beyond their own query
language.</p>
<p>Which brings me to how it is that you can develop and test an app for security devices and environments you don't
personally have access to. Actual sample log data is not easy to come by. And even if you do find it some Splunk apps
expect the log data to be ingested in particular ways (e.g. syslog), and replaying them from a file on disk is not
always trivial. </p>
<p>This is a concern both if you are writing a new app that will be used by others, or if you want to ensure that your
internal organization's apps are able to correctly handle log entries that are infrequent or that you might not
have encountered before. For example, the
<a href="https://docs.splunk.com/Documentation/CIM/latest/User/IntrusionDetection">Network Intrusion CIM model</a>
does not specify a finite list of possible values for the <code>action</code> field. And until you see some test examples you
won't know that the Check Point OPSEC TA field extractions will have the service name instead of the port number in the
<code>dest_port</code> field of the
<a href="https://docs.splunk.com/Documentation/CIM/latest/User/NetworkTraffic">Network Traffic CIM model</a>. So you'll need to
get a representative set of events and find what out values the TAs actually generate if you want to ensure your content
will adequately survive its first encounter with with real events out there in the wilderness.</p>
<p>Looking into this for the <a href="https://splunkbase.splunk.com/app/2777/">Niddel app</a> I stumbled upon a very useful tool
called <a href="https://github.com/splunk/eventgen">eventgen</a>. Basically what I have found is that most Splunk-developed apps,
and many developed by third parties, are shipped with configurations that allow you to easily generate sample log data
that you can then use to test your content. </p>
<p>The <a href="https://github.com/splunk/eventgen/blob/develop/README/Tutorial.md">eventgen tutorial</a> is very thorough and the
tool is very well documented. However the documentation is almost exclusively focused on app developers which want to
support event generation in their apps. Very little is said about how to simply use the functionality the app developers
already built-in. So let me try to help fill in the gaps here with the TL;DR version, which I only recommend for
development-only Splunk instances:</p>
<ul>
<li>
<p>First, confirm if the particular app or TA you are interested in has that configuration: go to its directory
under <code>$SPLUNK_HOME/etc/apps</code> (or download it from <a href="https://splunkbase.splunk.com">SplunkBase</a> and check within the
archive) for a <code>samples</code> directory and a <code>default/eventgen.conf</code> file. If it's there, you're in luck.</p>
</li>
<li>
<p>Then, ensure that the apps or TA for which you want to generate events os configured to share their content globally,
or the eventgen app (installed later) won't be able to access its sample data and configurations. The simplest way to
do this is on Splunk Web is to navigate to <code>Apps</code> -> <code>Manage Apps</code> and click on <code>Permissions</code> under the <code>Sharing</code> column
for the app you want events from. Make sure the <code>Sharing for config file-only objects</code> option is set to <code>All apps</code>:
<img alt="Splunk app permissions" src="https://www.infoseczanshin.com/images/splunk-test-data/app-permissions.png"></p>
</li>
<li>
<p>Download the <a href="https://github.com/splunk/eventgen/">eventgen GitHub repository</a> as a
<a href="https://github.com/splunk/eventgen/archive/develop.zip">ZIP file</a> and install it as a Splunk app in <code>Apps</code> ->
<code>Manage Apps</code> -> <code>Install app from file</code>.</p>
</li>
<li>
<p>The new eventgen app does not have a setup page when you go to it under <code>Apps</code>, just statistics on the generated events.
So the only way I found to enable and disable it is to go to <code>Apps</code> -> <code>Manage Apps</code> and click on <code>Set Up</code> on the row of
the eventgen app. Mark and unmark the <code>Enable ./bin/eventgen.py</code> checkbox to enable/disable event generation.</p>
</li>
</ul>
<p>Once this is done, eventgen will inspect all other installed apps it can access (hence the permissions step above) and
generate fake events for them according to their configurations, usually directly to the <code>main</code> index. No easy way to
choose which apps to generate events from at this time, unfortunately.</p>