Showing posts with label AWS. Show all posts
Showing posts with label AWS. Show all posts

Wednesday, August 19, 2015

Powershell: Automating AWS Route 53 Record Sets

Speaking of Cloud Environment where auto - scaling, self-healing servers have replaced the more conservative way of scaling – We can’t have classic manual way of DNS management.  I.e. Each new instance launched by auto scaling group should assign itself a human-readable hostname. In these cases Instance itself should be equipped enough to add its corresponding entry under Route 53. We can make use of Route53 and PowerShell to impalement this automation from Windows machine.

What Is Route 53?

Route 53 is a complete DNS solution provided by Amazon that lets you control every aspect of the name resolution process for your domain. You choose a registrar to reserve your domain, as common practice they often be managing all the DNSs for your domain. But if you are looking for more granular control over this Route 53 is one of the services you should be interested in. A details description of Route 53 is beyond the scope of this article, but you can check at AWS Route 53

Automation Script

I have wrapped the whole script into a function.We have used PowerShell V3 , consider that as prerequisite.Having Amazon cmdlets loaded should be done before expecting it to work.Pointers related to it's functioning :
  • Execution of this script will only materialize given working pair of Secret Key & Access Key
  • It validates the Record type with values mentioned under ValidateSet
  • If wait is specified, the function will wait until AWS confirms that the change is in sync. To be more specific it verifies whether entry has been propagated under specified Zone. By default i have kept it as false
  • If you have mentioned wait as true, waitinterval specifies the interval in milliseconds between each polling
  • Every parameter can be overridden why calling this function
Function Create-AwsRoute53Record
{
    [CmdletBinding(SupportsShouldProcess=$true)]
    Param (
                [String]$AccessKeyID="****************",
        [String]$SecretAccessKeyID="*****************",
                [String]$Region="us-east-1",
                [Parameter(Mandatory=$true)] $Zone,
               [ValidateSet("A", "SOA", "PTR", "MX", "CNAME","TXT","SRV","SPF","AAAA","NS")]
                [String]$RecordType,
               [Parameter(Mandatory=$true)]
                [String]$Name,
              [Parameter(Mandatory=$true)]$Value,
               $TTL,
               $wait = $false,
              [int]$waitinterval = 1000

        )
   
    Process
    {
     Set-AWSCredentials -AccessKey $InfoObject.AccessKey  -SecretKey $InfoObject.SecretKey
    Set-DefaultAWSRegion -Region $region
    $zoneEntry = (Get-R53HostedZones) | ? {$_.Name -eq "$($Zone)."}
 
$hostedZone = $zoneEntry.Id
        if (@($zoneEntry).count -eq 1) {
                $Record = new-object Amazon.Route53.Model.ResourceRecord
                $record.Value = $Value
                 #add the trailing dot
        if (!($Name.EndsWith(".")) -and $Name)
            {$Name += "."}

        $ResourceRecordSet = New-Object Amazon.Route53.Model.ResourceRecordSet
        $ResourceRecordSet.Type = $RecordType
        $ResourceRecordSet.ResourceRecords = $Record
        $ResourceRecordSet.Name = $Name
        $ResourceRecordSet.TTL = $TTL


        $change = New-Object Amazon.Route53.Model.Change
        $change.Action = "Create"
        $change.ResourceRecordSet = $ResourceRecordSet
     
        $Changes = (New-Object -TypeName System.Collections.ArrayList($null))
        $Changes.Add($Change)

        Try
        {
            $result = Edit-R53ResourceRecordSet -ChangeBatch_Changes $Changes -HostedZoneId $hostedZone
         
        }
        Catch [system.Exception]
        {
            Write-error $error[0]
        }
     
        if ($result)
        {
            if ($wait)
            {
            #Keep polling the result until it is no longer pending
            Do
                {
                    #get change status
                    if ($SecondPoll)
                        {Start-Sleep -Milliseconds $waitinterval}
                   $status=Get-R53Change -Id $result.Id
                    $SecondPoll = $true
                    Write-verbose "Waiting for changes to sync. Current status is $($status.Status.Value)"
                }
            Until ($status.Status.Value -eq "INSYNC")

     
            }
     
            $Status
        }
     }
  }
}

Script Execution

Below we have a sample execution which captures the Public IP of AWS instance ,  and subsequently add it to specified Zone as A Name record Type.

# Change the values as per your specification i.e Zone , Name , RecordType and TTL. Wait is enabled to provide added control.
$LocalIP = Invoke-RestMethod -Uri http://169.254.169.254/latest/meta-data/local-ipv4 -Method Get
$LocalIP=$LocalIP.Trim()
Create-AwsRoute53Record -Zone xyz.com -RecordType A -Name testing.xyz.com  -Value $LocalIP -TTL 500  -wait $true

Wednesday, August 5, 2015

Powershell: Automating AWS Security Groups

To provision and manage EC2-Instances in AWS cloud that comply with industry standards and regulations, Individuals administrating that should understand the security mechanisms within AWS framework—both those that are automatic and those that require configuration.Let’s take a look at Security Group which falls under the latter category.

As there is no "Absolute Security Group" which can be plugged in to satisfy the universal need, we should always be open for its modification.Automating so via Powershell will provide predictable/consistent results.

What Is Security Group?

Every VM created through AWS Management Console (or via scripts) can have association with one or multiple Security Groups (in case of VPC it can be up to 5). By default all the inbound and out bound traffic flow at instance level is blocked from elsewhere. We should automate the infrastructure to open only the ports satisfying the customer need. This implies that we should add rules to each Security Group for ingress/ egress as per customer requirement.For more details have a look at AWS Security Group

It is duly important to allow traffic only from valid source IP addresses; this will substantially prune security attack surface, use of 0.0.0.0/0 as IP range makes things vulnerable for sniffing or tampering of infrastructure. Traffic between VMs should always traverses through Security Groups, we can achieve this by allowing initiators Security Group- ID as source. 

Automation Script

I have kept this as a single block ,if one wishes they can create a function out of it. few things worth considering :
  • Execution of this script will only materialize given working pair of Secret Key & Access Key
  • This script make use of filtering functionality, whereby it expect end user to provide some Name-Pattern ,selection of Security Group is driven by aforementioned pattern
  • To facilitate the whole operation you have to provide certain parameters i.e.[IpProtocol , FromPort , ToPort , Source]
  • Source parameter can be interpreted in two ways, you can either provide IpRanges in CIDR block format or choose another Security Group as source in the from of UserIdGroupPair
<#
    .SYNOPSIS
    Simple script to safely assign/revoke Ingress Rules from VPC Security Group .
    
    .DESCRIPTION
    Script first checks to see what are the rules has beein specified for update,if already assigned will do no harm. 
    If assginement is successful, same can be verified at AWS console.
    
    NOTE:  Script must be updated to include proper pattern, security credentials.
#>

# Update the following lines, as needed:

Param(
          [string]$AccessKeyID="**********",
          [string]$SecretAccessKeyID="********",
          [string]$Region="us-east-1",
          [string]$GrpNamePattern="*vpc-sg-pup_winC*",
          [string]$GroupId="sg-xxxxxxxx",
          [string]$CidrIp="0.0.0.0/0",
          [switch]$SetAws=$true,
          [switch]$Revoke,
          [switch]$Rdp=$true,
          [switch]$MsSql=$true
         )

     $InfoObject = New-Object PSObject -Property @{
        AccessKey = $AccessKeyID
        SecretKey = $SecretAccessKeyID
        Region=$Region
        GrpNamePattern = $GrpNamePattern
        GroupId=$GroupId
        CidrIp=$CidrIp
}
if($SetAws)
    {
 Set-AWSCredentials -AccessKey $InfoObject.AccessKey  -SecretKey $InfoObject.SecretKey 
 Set-DefaultAWSRegion -Region $region
    }
       $PublicGroup = New-Object Amazon.EC2.Model.UserIdGroupPair
       $PublicGroup.GroupId= $InfoObject.GroupId

       $filter_platform = New-Object Amazon.EC2.Model.Filter -Property @{Name = "group-name"; Values = $InfoObject.GrpNamePattern}
       $SG_Details=Get-EC2SecurityGroup -Filter $filter_platform |SELECT GroupId, GroupName

       $rdpPermission = New-Object Amazon.EC2.Model.IpPermission -Property @{IpProtocol="tcp";FromPort=3389;ToPort=3389;UserIdGroupPair=$PublicGroup}

       $mssqlPermission = New-Object Amazon.EC2.Model.IpPermission -Property @{IpProtocol="tcp";FromPort=1433;ToPort=1433;IpRanges=$InfoObject.CidrIp}
       $permissionSet = New-Object System.Collections.ArrayList

     if($Rdp){ [void]$permissionSet.Add($rdpPermission) }

if($MsSql){ [void]$permissionSet.Add($mssqlPermission) }

   if($permissionSet.Count -gt 0)
  {
   try{
if(!$Revoke){
      "Granting to $($SG_Details.GroupName)"
       Grant-EC2SecurityGroupIngress -GroupId $SG_Details.GroupId -IpPermissions $permissionSet 
 }
  else{
  "Revoking to $($SG_Details.GroupName)"
  Revoke-EC2SecurityGroupIngress -GroupId $SG_Details.GroupId -IpPermissions $permissionSet
}
    }
catch{
if($Revoke){
Write-Warning "Could not revoke permission to $($SG_Details.GroupName)"
}
          else{
Write-Warning "Could not grant permission to $($SG_Details.GroupName)"
   }
    }
}


what we are looking at being able to automate Creation/updation of Security Group.Use this script in case you ran into frequent changing of Security Groups.

P.S. This script has been written keeping VPC  in mind, Different parameter usage between VPC and EC2 security groups should be take care of.

Saturday, March 14, 2015

DevOps with Windows - Chocolatey on AWS


Conceptually Package manager is well understood space for someone having slightest understanding of how *nix environment get’s managed, but when it comes to windows it was untracked space till recently. This was piece of stack which was ironically missing for so long that once you get hands on you will feel how on earth you were living without it. NuGet and Chocolatey are the two buzz words making lots of noises and deemed as the future for windows server management.

What Is Chocolatey?


Chocolatey builds on top of NuGet packaging format to provide a package management  for Microsoft Windows applications , Chocolatey is  kind of yum or apt-get but for windows. Its CLI based and can be used to decentralize packaging.  It has a central repository located at http://chocolatey.org/.

If you have ever used windows build in provider you probably be aware of the issues it has. It doesn’t really do versioning and seems misfit for upgrading. Any organization looking for long term solution to ensure that latest versions are always installed for them build in package provider may not be the recommended option .Chocolatey takes care of all this with very little effort. In contrast to default provider which has no dependency Chocolatey requires your machine to have Powershell 2.0 & .Net framework 4.0 installed. Installation of packages from Chocolatey is one command line that reaches out to internet and pulls it down. That would be version-able and upgradable; you can specify this version of package and that that’s what gets installed.

Recommended way of Chocolatey installation is by executing PowerShell script.

PS:\>iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))

Chocolatey With AWS


AWS offers windows instances with both their offering; under IAAS you can launch windows instance as EC2, whereas with PAAS you can get that via Elastic beanstalk. 

Using Cloud Formation:

Using ‘cfn-init’ AWS Cloud Formation supports the download of files and execution of commands on Windows EC2 instance. Bootstrapping of Windows instance using Cloud Formation is lot simpler than any other ways. We can leverage this offering to install Chocolatey while launching the server using CFT.While doing this using Cloud formation we have to execute PowerShell.exe and provide the install command to that.One thing to be take care of that Chocolatey installer and the packages it installs may modify the machine's PATH environment variable. This adds complexity since subsequent commands after these installations are executed in the same session, which does not have the updated PATH. To overcome this, we utilize a command file to set the session's PATH to that of the machine before it executes our command. We will create a command file 'ewmp.cmd' to execute a command with the machine's PATH, and then we will proceed with Chocolatey and any other installation. With below sample we will be installing Chocolatey and then install Firefox with Chocolatey as provider

"AWS::CloudFormation::Init": {
  "config": {
    "files" : {
      "c:/tools/ewmp.cmd" : {
        "content": "@ECHO OFF\nFOR /F \"tokens=3,*\" %%a IN ('REG QUERY \"HKLM\\System\\CurrentControlSet\\Control\\Session Manager\\Environment\" /v PATH') DO PATH %%a%%b\n%*"
      }
    },
    "commands" : {
      "1-install-chocolatey" : {
        "command" : "powershell -NoProfile -ExecutionPolicy unrestricted -Command \"Invoke-Expression ((New-Object Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))\""
      },
      "2-install-firefox" : {
        "command" : "c:\\tools\\ewmp choco install firefox"
      }
    }
  }
}

Using AWS Elastic Beanstalk:

AWS Elastic Beanstalk supports the downloading of files and execution of commands on instance creation using container customization. We can leverage this feature to install Chocolatey.
The aforementioned installation can get translated into AWS Elastic Beanstalk config files to enable use of Chocolatey in Elastic Beanstalk. The change while doing using Elastic Beanstalk; we will create YAML .config files inside the .ebextensions folder of our source bundle.
files:
  c:/tools/ewmp.cmd:
    content: |
      @ECHO OFF
      FOR /F "tokens=3,*" %%a IN ('REG QUERY "HKLM\System\CurrentControlSet\Control\Session Manager\Environment" /v PATH') DO PATH %%a%%b
      %*
commands:
  1-install-chocolatey:
    command: powershell -NoProfile -ExecutionPolicy unrestricted -Command "Invoke-Expression ((New-Object Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))"
  2-install-firefox:
    command: c:\tools\ewmp choco install firefox

Above will work in the same way as cloud formation sample did , it will Create a command file 'ewmp.cmd' to execute a command with the machine's PATH before installing Chocolatey and Firefox.

P.S. Chocolatey can best be used as package provider for puppet on windows. Puppet offers great support in promotion and execution of Chocolatey on Windows.