Flip a couple of switches in your AWS console, right? Wrong! It is so much cooler than that!

In exercise 5.3 of the AWS CSA Associate exam, there is an exercise which really wants you to stretch your wings. That is what I love about this book; getting the most out of this book requires you to use resources outside this book.

The goal of the exercise is to import custom metric sources from your EC2 instance that are not available from the CloudWatch console. What is interesting is that I only have a few instances across my AWS ecosystem, but I already have a choice of 200 metrics to dig around in, mostly from the EC2 namespace. My first thought runs to the kinds of tool sets that you will need to be running to make sense of all the data in your average Enterprise AWS environment…

First things first: Second things come later, if ever.

1. You need to install a few Perl libraries: libwww-perl and libdatetime-perl. Yep, the AWS guys opted to use Perl to bridge the gap between your virtual boxes and their hypervisor. Cool.
2. Download the scripts package from Amazon here:https://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.2.zip
3. Grab your keys from a user whose role is either admin or has the appropriate cloudwatch:put/get/listmetrics permissions. The EC2:DescribeTags permission might be nice too. Take those keys and insert them into the awscreds.template file and then change the extension from template to conf.
4. Boom! You’re ready to go. I fired up this agent by using the ./mon-put-instance-data.pl -mem-used-incl-cache-buff -mem-util -mem-used -mem-avail command, and it gives me all the memory usage data points that I could ever want. At this point, you will find them in the metrics section of your CloudWatch console.
5. A best practice would be to set a cron job for a much more frequent update by editing the crontab file and setting in this:*/5 * * * * ~/aws-scripts-mon/mon-put-instance-data.pl –mem-used-incl-cache-buff –mem-util –disk-space-util –disk-path=/ –from-cron
   

Most of this is documented very thoroughly on the AWS Docs site here, but it will lack the motivational empowering rhetoric and sound effect footnotes.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html

As HTTP constantly evolves, so do the security concerns around it. I’m compiling a list of resources that I have been using in my study and reiterating some of the key concepts. Forgive the muddle as the organization of this article takes shape.

1.  Security operates at multiple layers of an app:

– Application code
– Application configuration
– Web Server configuration
– Application Firewall
– Dependent hosts
– Client Side Security Controls ( browser sandboxing? )
– Logging Monitoring Alerting
- Deceptive Defense

2. Client Side Security Control example:

HTTP Response Headers instruct browsers to validate untrusted input, prevent executions and report incidents. Here they are:

– HSTS (RFC 6797) — good reading!!
– Public Key Pinning Extension ( HPKP )
– X-Frame-Options
- X-XSS-Protection
– X-Content-Type-Options
– Content-Security-Policy
– X-Permitted-Cross-Domain-Policies
– Referrer-Policy
– Expect-CT

3. XSS remediation in its entirety is a pipe dream. There are so many JS components that you will be using that can present ways for input to get into the app. This is why CSPs are used.

4. CSPs function is a few ways:

– Only load resources from current origin or domains listed in policy.
– Restrict by protocol, domain and path.

The issue with having a restrictive CSP is that in many cases, your bootstrapped code and javascript libraries are making calls to other sites for their functionality. Setting up a whitelisted CSP can break your site. Are your javascripts loading additional resources? You would need to whitelist them.

Checkout CSPvalidator.org: This is what I pulled back from www.yahoo.com:

content-security-policy: sandbox allow-forms allow-same-origin allow-scripts allow-popups allow-popups-to-escape-sandbox allow-presentation; report-uri https://csp.yahoo.com/beacon/csp?src=frontpage&site=fp&region=US&lang=en-US&device=desktop&partner=default;

Interesting directives in there: allow-popups-to-esapce-sandbox

I wonder what the ramifications for this could be? Thats an exploration for another day….

5. HSTS: a web security policy mechanism that helps protect websites against protocol downgrade attacks and cookie hijackjing. It automatically turns insecure links into secure ones. If the security of the connection cannot be ensured ( such as the cert is bad ), the connection is terminated. HSTS — jeez.. this could get its own learner section: turns out there is something called a ‘max-age Directive’ which specifies a time limit that the user-agent considers the responding host as a ‘Known HSTS Host’. HSTS has the capacity to fix SSL stripping MITM ( Moxie Marlinspike ).

HSTS headers can be stripped away by an attacker if it is the users first visit to a site. Some browsers have attempted to solve this problem by including a pre-loaded list of HSTS sites but this will not scale to the whole internet. HSTS can also help prevent session hijacking or credential theft by tools such as firesheep.

max-age
includeSubdomains
preload
TOFU problem – trust on first use

https://hstspreload.org/

5. Content Security Policy: 

A powerful header that aims to prevent XSS and data injection attacks by restricting JS and DOM execution elements. 

Declared by Content-Security-Policy or Content-Security-Policy-Report-Only

.

Requires careful tuning

, currently there is v1, v2, and v3.

Directives:

*-src : Define valid source of JS, images, CSS, etc.
nonce-* / sha256- (v3): Only allow is SHA value matches
strict-dynamic: Allow document.createElement(‘script’)

This article will continue to grow in scope and size. Please return on occasion to see how it gets built out.

——————-

——————-

Resources:

1. HTTP RFC: https://www.ietf.org/rfc/rfc2616.txt
2. HTTP Mozilla Reference: https://developer.mozilla.org/en-US/docs/Web/HTTP
3. HTTP Security Talk by Pedram Hayati of Ettlam https://www.youtube.com/watch?v=ZZUvmVkkKu4
4. Web Application Hackers Handbook

XAMPP — The cool localhost web server that has everything you need to test webservices – is a bit tricky to setup. Guidance from this page https://github.com/snoopysecurity/dvws gets you started. Here are the things to remember:

You have to clone the DVWS repo into the htdocs folder that XAMPP sets up during the install. When you have a successful install and you have gone to localhost:8080 and you see the pretty apache friends landing page, you are reading the index.php page that is in the htdocs folder.

Cloning the DVWS project into that folder is the tricky part. Where is the htdocs folder? How are we going to clone into it if we cant find it on the CLi? ( Always use the CLi. Its the rule… ) If you hit the explore button, it will open up the folder in the finder and then you can mav-Up from there which will get you to the virtual mount next to your HDD, but dragging the htdocs folder onto the terminal window reveals the complete path that you can use — filesystem access is granted through a hidden folder in your home directory — .bitnami

Seeing the /opt/lampp path was the lightbulb. There is no opt directory on the mac file system. This is a mounted drive that will be accessed through a virtual mount in your home directory. If you are not familiar with bitnami installations, this has a chance of holding you up.

Management: I will supply you with an AWS account and Ubuntu Web Server in an unconfigured state. You will be responsible for uploading the content and taking it live to the public facing IP. Once the site is live and tested on the major platforms/devices/browsers the project will close.

Frameworks: No proprietary frameworks or dependencies are to be used in the development of the site. Your code will need to work on a LnMP stack ( Linux, Nginx, MySQL, PHP ) as that is the web server that I will be building and the environment that WordPress runs on.

Testing: Functional Testing needs to be performed on the major platforms: Windows 10 [IE, Chrome, Firefox], MacOS High Sierra [Safari, Chrome, Firefox], iOS [Safari], Android [Chrome]. Site needs to render without error, display uniformly according to design, render mobile version where appropriate [hamburger must function smoothly ], and allow smooth navigation in each instance.

WordPress: I will setup the wordpress instance but the design will be required to fit around it. Once the wordpress instance is up and live, you will take the design reigns. 

Design:  Front page, About page, Projects page, 404 page, Mobile rendering: I will supply you with the graphics, a general design plan and you will create a mockup via a graphics program and I will review it before we execute on the final design. You will allow for up to three iterations. The code will be written from scratch ( or at least independent of frameworks ) such that it will have portability independent of any platform technologies like Ruby, for example. Navigation will be available across a top-bar, down the side and on the bottom. CSS font layouts are TBD. To create the mockup, we will do a screenshare of some type and use a whiteboard to transmit ideas for how the site will look.

Design of the site will follow these sites:

Danielmiessler.com – [ overall styling, this is a good simulacrum, but my site will have no front page like this one… ]
http://docs.python-guide.org/en/latest/dev/virtualenvs/  [ use font from this site ] – no black topbar though… 

Image will go in top right hand corner. That image will represent 25% of the width of the page. The remaining 75% will have the title of the site in a top bar that you create. Use your own design idea — the title of the site is “The Secure Method”. Just type the words in and frame it up.

The frame of the site will be divided up along the left edge of the corner graphic. Below the corner graphic will be a side bar that will have other content that has not been finalized yet. Please use the graphic sent to Bill for reference. The background will be plain white.

please visit the above sites to get a feel for the design elements that have been assumed.

External API calls/ Analytics Frameworks / Dependencies: NO! Boo!!!!! If I am on the box and I pull up the site using http://localhost and the box has no internet connection, the site should fully render. There will be no dependencies built into the site at all.

Extensibility: I will need to be able to add pages to the site and respective navigation modules as they are needed independent of you. Part of this project will be to create a page template that can be duplicated and titled as needed with instructions for how to add a reference o the new page into the navigation as needed.

Documentation: I will need a written artifact describing the design of the site. A few paragraphs will do Im sure.

Final Deliverable: Final delivery will be achieved once the site is up on the aws instance and platform testing has been verified.

Anytime you are presented with clickbait, you must respond to it like the garbage that it is, seeking to rob you of your attention for the precious few cents a marketer will get from the thousands that are ‘activated’ from a particular ad being deployed en masse.

If you’re on youtube or some other popular content portal that can self deploy and manage its own marketers ( and not be suppressed by the popular Ad Blocker browser plug-in or equivalent app for your mobile device ) and you see a suggested link that is labeled like the list below, roll your eyes and send bad juju to the owner of said content:

• 10 ways that blah blah does blah blah blah
• You will not be the same after reading this!
• If this wasn’t on video, you wouldn’t believe it!
• AMAZING!!! ( or anything else in all caps ) 
• 💚 — Anything with the latest and greatest emoji that has been given unicode representation to now attract your eyes… 

This is the web at its worst. Guard your attention carefully and take steps to build in a discipline that prevents this kind of mental infection. It spreads like a disease without you even knowing it. Install Purify in your iPhone. Install the equivalent in your Android. Then attack yourself for owning an Android.

My guess is the Las Vegas shooter was suicidal in a very weird way. Something in him snapped, but it snapped a long time ago, and the amount of time it took for him to plan this and carry it out is an awkwardly long time to be suicidal. There was contempt and hate fueling his death march to the Mandalay Bay. Sometimes when I try to understand a person’s motives, I play a little game. What is the worst reason he could have had for doing this? Insanity? Russia? If we could only have gotten a snapshot of his psychopathology and synaptic condition the last few weeks leading up to the night of the concert…

Large crowds are the prime target for anyone who wants to kill a lot of people. And they can do so with relative ease from a high position using an automatic rifle, especially against a crowd at a concert where the saturation density of people can become so high that it prevents exfiltration of those who wish to leave in a hurry.

It is damn amazing that more people didn’t die in Las Vegas. I’m rather astounded based upon the fact that the gunman was firing from two different vantage points, each position having interior cover. Ive heard an estimate from a news media outlet that it took nine minutes for a group of police officers to assemble, figure out where the shooting was coming from, converge on the hotel and neutralize the shooter.

9 minutes.

Not 19. Not 90. Not long enough for him to expend all of his ammo. 9 minutes. That is amazing response time. And there was also a great coordination response from the Mandalay Bay security personnel. It would seem that LVPD and localized security teams have actually spent some time preparing for this kind of thing, and they have done so with diligence and dedication. My hats off to the Sin City badges and the hotel patrols.

Now its time to get that response time to come down even lower. I think that we need to start protecting these large musical events with active firepower. Its time to assume that someone is going to show up and start shooting. The security at these events needs to resemble the preparation of an event that will be attended by the President. When the POTUS is going to make an appearance, the Secret Service show up in advance and do a risk analysis of the entire area, looking for any place a shooter could position himself. They lock the area down, perform area-denial of all places advantageous to a combatant and post sharp shooters in every direction waiting for someone to get stupid.

Understandingly, there are more risks involved with using rifles to keep people from using rifles around large crowd, but I think these risks can be managed. It is worth the risk, planning and effort to keep a well trained and well focused team at the ready. Had the shooter had a few expert marksman to keep him busy the moment that he broke out the glass of his 32nd story window, there would have been fewer casualties.

And there is plenty of money in the budget for these events to pay for it. How much do we pay for Cochella and Outside Lands? Yeah… the margins are there to the point that this protection doesn’t have to be passed on to the consumer.

I attend concerts like this one all the time. I’m still going to. Shoot at me all you like. You are not going to stop me from seeing my favorite band. I just hope the promoter has somebody ready to shoot back at you when the time comes.

xargs is cooler. Thats why…

In trying to delete a bunch of files with similar names  ( REALLY necessary when playing around with the split command ) but there usually is an issue — which presents an inconsistent problem. Sometimes linux barfs and sometimes it behaves…

When issuing this command:   find . -name ‘someFile*’

I will get the output:

./someFileA
./someFileB
./someFileC
etc….

And this kind of input being piped to an xargs rm command doesnt jive all that well. So there is cause for use of the exec command… but its very unwieldy in its usage:

find . -name ‘someFile* -exec rm {} \;

Why is it unwieldy? You mean besides the fact that I wanted to say the word unwieldy? Well, I just don’t understand the syntax. Lets explore:

-exec: acts as a pipe in this instance, but the mechanism is different and those differences are subtle.

the {} seems to act as a transformation to an iterative for each string returned separated by a new line delimiter. It seems to hold true to its Array symbolic reference.

Not sure about the \ and the ; and the option of the +. I will have to look that up later.

1. Make sure that your bank requires that you tell them your ATM card number when you call in. If they don’t, tell them to add that and other layers of security to your account. Be worried that your money will evaporate.
    
2. Call your mobile phone service provider and make sure you they ask you for a PIN. AT&T does this already but I’m not sure about Verizon or Virgin Mobile, etc.
3. Enable two factor authentication on all of your accounts.
4. Keep close tabs on, or even FREEZE your credit with Experian, TransUnion and the spectacular producer of this fine mess. Its free. It may be hard to get through to Equifax, though. They might be a bit swamped.
5. Understand that you are highly vulnerable to being impersonated on a variety of levels now. 
6. If you have an Equifax account and that password is shared with any other of your accounts, change that password. Then smack yourself for sharing passwords among accounts.
7. If you go to the emergency website set up by Equifax to see if you have been breached, attack yourself and don’t do that. Equifax is poison and you’re just submitting more information to the hackers. If you sign up for their credit monitoring service, this abdicates you from joining the class action lawsuit against them where I imagine that they will be liable for more money than they are worth as a company. If you sign up for credit monitoring, don’t do it through them.Ugh. I was wondering why my credit score was so high..
   Gods help us all. Make no mistake, Equifax has caused severe harm to the entire country. They have put the sheep into the hungry wolves den. Now a bunch of NASA engineers are going to have to sit around and figure out ways for people to prove that they are themselves without using any of the exposed information. Yeesh..I bet its also time to knock on Experian’s door… and TransUnion’s as well and see if they have been leaving their doors unlocked at night…

A journal entry for fundamental lessons that advance programming fluency:

1.  What a return statement does:

   When a function ‘returns’ a value, it doesn’t go anywhere in particular. What is special about a value that is returned is that it is now available to be called by other methods and functions.

   For example, here’s a function utilizing both print() and return:

   def foo():
       print("hello from inside of foo")
       return 1

   Now you can run code that calls foo, like so:

   if __name__ == '__main__':
       print("going to call foo")
       x = foo()
       print("called foo")
       print("foo returned " + str(x))

   If you run this as a script (e.g. a .py file) as opposed to in the Python interpreter, you will get the following output:

   going to call foo
   hello from inside foo
   called foo   
   foo returned 1

   Logical example supplied by Nathan Hughes.

2.  How recursion works: 
   If you define a small script with a nested return statement, the untrained logic will present a computational fallacy. Note the example:

   def f(n):
   …     if n<2:
   …             return 1
   …     return f(n-1) + f(n-2)
   x = f(3)

   Print x

   How will this code work? The tough spot is that this algorithm is especially designed to fool you. If you calculate the computation algebraically, you get the same answer as you would have if you understood where recursion was supposed to take place — in this case using the integer ‘3’. Take a moment to run the numbers in your head.

   Now, If you were to shove in the number 2, different devils come out of the details. Try coding this up in a Python interpreter and see what you come out with.

   3 gets you 3. 4 gets you 5. But 2 gets you 2, not 1. Why do you think that is? Understanding recursion means that when you get to the second return line, you understand that each f(someValue) must be looped back up through the first if statement again and be subject to satisfying the first return statement again.

And here’s the thing.. I had it working before… doesn’t that just kill? The irony is starting to hurt my feelings… 🙂

After spending about 7 hours trying to get this to work, I have decided to start over and spin up a fresh install of the latest asg*.iso file in vmware and start there and see how it goes….

Okay… its been 6 hours and I have a fresh install working and configured for baseline internet access using one network client ( manually adjusting TCP stacks away from DHCP to test) and we have pulled our standard test http blob. ( Adell’s ‘hello’ video on youtube.. my version of hello world )

Pain points: Spinning up a twin firewall in ESXi can be problematic so remember these things:

( im going to have to condense these into simple captures of what I learn instead of talking to myself.. )

• Go for the Other Linux 32 bit guest framework in ESXi for Sophos UTM. There is an option mid-install to go for the 64 bit, but for some reason the WebAdmin always chokes right after install when you use the 64 bit option. Just pass on it.
• Twinning up a firewall with the same IP address ( even with the other one down ) seems to make the gods angry. I didnt test for an ARP cache conflict, but Im guessing thats what it was.. no workie. Use a new IP.
• Don’t worry about configuring a gateway in the Firewall. It auto configs.
• When setting up your WAN interface, choose Standard ethernet as your connection type and opt for DHCP. ( Much better performance than our first exercise in building this… )Now… time to set up the VPN. But steaks first. ( Will return )

Next Day —- The bleeding continues…
9:13AM

I have another client on the network configured to run through the new firewall. Im configuring the SSL VPN now…. wish me luck.

Interesting tidbit: Nmap has a DNS dependency when running from Darwin 15.6 without using a -Pn flag. Weird….

Toggling the Firewall Rules has no effect over the course of 1-2 min of blocking access. I turned them off yet I can pull websites and my DI stream does not fail. — Does it take a few mins? Need to explore further to elaborate on root cause of firewall issues… Fleh.

Some great nmap commands listed here:

http://bencane.com/2013/02/25/10-nmap-commands-every-sysadmin-should-know/

Thank you Ben Cane. 🙂

3:45PM
Yah know… its been another 6 hours of nibbling along and my only accomplishment today is making my bed.. Redoing the firewall from scratch has forced me to learn every detail of this firewall down to where the developers went to kindergarten and what level their Dungeons and Dragons character is. — totally relevant I know… but not the short answer to the issue at hand.. being able to push code to my home web server… all this firewalling for some python and flask….

rrrrrrrr… :\

Working on getting the DMZ access to the internet… did I enable masquerading?…

4:22
This must be what its like when the zombie is trying to get out of the grave and there are several townspeople all weighing down on top of the dirt and coffin lid that is pushing through the ground…a rigor’d arm with bones and flesh all rotten pushed through the ground… lol

So now I have internet access out of the DMZ and into the interwebs:
~$ wget danielmiessler.com
–2016-10-16 16:20:50–  http://danielmiessler.com/
Resolving danielmiessler.com (danielmiessler.com)… 104.25.35.29, 104.25.34.29, 2400:cb00:2048:1::6819:231d, …
Connecting to danielmiessler.com (danielmiessler.com)|104.25.35.29|:80… connected.
HTTP request sent, awaiting response… 301 Moved Permanently
Location: https://danielmiessler.com/ [following]
–2016-10-16 16:20:53–  https://danielmiessler.com/
Connecting to danielmiessler.com (danielmiessler.com)|104.25.35.29|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: unspecified [text/html]
Saving to: ‘index.html.1’

Im using my local DNS but still it had to do a double request to get to the site now that Dan has put a cert on his page. Shouldnt the entry default me to the 443? Maybe its an Ubuntu thing..

9:20

Grrrrrrr. Still no workie. Same problem that I had with the other box. Same errors reporting from Viscosity but with no viable description of whats wrong…

Viscosity lacks Verbosity. Im downloading PFSense….

–Update 20 April 2017…

I did it. Finally. The culprit? Comcast’s modem. The new model that they switched in when I moved the office came with a Un-Switch-Off-able firewall that I had to open up a port on in the silliest, non-intuitive way, specifying the route NAT’d from the modem to the External WAN interface in Sophos. — that took me a while… Snarbs!

Its a real victory being able to come back to a problem over and over again determined to crush it and then finally doing so… Especially when you already had the gold in your hands before…