Pages

Sunday, July 7, 2019

AWS CLI - Configuring the Command Line Interface for Aws

AWS CLI is an open source tool from Aws that enables Aws platform users to communicate and issues commands on various services of the Aws. This tool lets users to issue commands which in turn invoke the aws services via APIs exposed by those services. The authentication and authorization of the user issuing the commands are abstracted by configuring the Cli with the users credentials or keys. 


The Cli provides higher level of operations on the services as simple services to the users. By having this Cli, users can now create shell scripts or automate simple and complex tasks on the services.


Installation - Installing Cli is quite easy. Though there are few prerequisites like having a python boto library. We can install the Cli using python pip as “pip install awscli”.


Configuration - Once we have the tool available we can run the “aws” command as below,
jagadishm@[/Volumes/Work]: aws
usage: aws [options] [ ...] [parameters]
To see help text, you can run:

  aws help
  aws help
  aws help


Configure the access_key_id and secret_access_key from IAM -> Access keys -> Generate. This will create a keyId and an access key. 


Once a Key id and access key are generated, we will now configure the aws cli as,
jagadishm@[/Volumes/Work]: aws configure
AWS Access Key ID [None]: AKIAIHJJ6T4NSENA4M3A
AWS Secret Access Key [None]: **********
Default region name [None]: us-east-2a
Default output format [None]: 


Configuration files
Once we configure the aws cli, 2 files will be created in the ~/.aws directory.

~/.aws/config - This contains configuration of multiple named profiles including the default profile. The configuration of each profile consist the key/value parts for a region and its output properties.
jagadishm@[~/.aws]: cat config 
[default]
region = us-east-2a
output = json


~/.aws/credentials - This file contains the credentials of multiple named profiles including the default one. The credentials are available for each profile that we have in the config file.
jagadishm@[~/.aws]: cat credentials 
[default]
aws_access_key_id = AKIAIHJJ6T4NSENA4M3A
aws_secret_access_key = *****************


Named Profile - Aws cli supports using named profiles. The named profiles are nothing but profile which has a specific name and aws cli can use this name when invoking services in aws. A named profile can be created as,
jagadishm@[~/.aws]: aws configure --profile admin
AWS Access Key ID [None]: AKIAIHJJ6T4NSENA4M3A
AWS Secret Access Key [None]: *************
Default region name [None]: us-east-2a
Default output format [None]: json


Now if we go to the ~/.aws/config, we can see a new profile with the name created as below,
jagadishm@[~/.aws]: cat config 
[default]
region = us-east-2a
[profile admin]
output = json
region = us-east-2a


List Config Data - Aws cli allows us to list the data available using,
jagadishm@[~/.aws]: aws configure list
      Name                    Value   Type Location
      ----                    -----       ---- --------
   profile                      None None
access_key     ****************4M3A     shared-credentials-file    
secret_key     ****************PPIr        shared-credentials-file    
    region               us-east-2a         config-file ~/.aws/config

jagadishm@[~/.aws]: aws configure list --profile admin
      Name                    Value Type   Location
      ----                    ----- ----   --------
   profile                    admin manual --profile
access_key     ****************4M3A shared-credentials-file    
secret_key     ****************PPIr shared-credentials-file    
    region               us-east-2a config-file    ~/.aws/config


Try out basic commands - Once that we have configured the aws cli, we can run a few commands as below,


List all s3 buckets in the current region of us-east-2a
jagadishm@[~/.aws]: aws s3 ls
2019-07-03 11:23:13 cf-templates-1kdx3hvziz90i-us-east-1
2019-07-03 11:26:35 sample-bucket-simplesamplebucket-1tuihgl33nnsy


Using Named profile
jagadishm@[~/.aws]: aws s3 ls --profile admin
2019-07-03 11:23:13 cf-templates-1kdx3hvziz90i-us-east-1
2019-07-03 11:26:35 sample-bucket-simplesamplebucket-1tuihgl33nnsy


Hope this helps in getting you started with Aws Cli.

Read More

Cloud Formation - Understanding the Template

In this second article of Cloud formation, we will understand some more details of the template and various sections that can be used in the template.

We already know that the definition of the infrastructure is done inside a template file. The template is a JSON format file that describes our infra. This template has to adhere the AWSTemplateFormat guidelines.

1. The AWSTemplateFormatVersionsection (optional) identifies the capabilities of the template.

  {
    “AWSTemplateFormatVersion”: “2010-09-09”
  }


2. Description allows to write comments for the template file that we are creating.
 {
    “AWSTemplateFormatVersion” : “2010-09-09”,
     “Description” : “example template” 
  }


3. Metadata allows us to include more details about the template. Additional details for the resources that we create will be added in this optional metadata section.

"Metadata" : {
  "Instances" : {
           "Description" : "this is for app servers"
    }
}

4. Parameter allows us to customise the template by passing values as inputs when we are creating the stack. This way we can pass different values each time when building the stacks. A parameter can be defined as,

"InstanceType" : {
      "Type" : "String",
      "Default" : "t2.micro",
      "Description" : "Type of EC2 instance to launch"
}

"KeyName" : {
     "Description" : "EC2 Key Pair for SSH Access
     "Default" : "sample",
     "MinLength": "1",
     "MaxLength": "64",
     "AllowedPattern" : "[-_ a-zA-Z0-9]*",
}

In the above snippet we are defined a parameter called “InstanceType”. The parameter type is the string which means we will be passing a string value as input when building the stack. The parameter supports multiple properties like providing default values, length, max and min values etc.

The above InstanceType parameter is string type and default value it t2.micro. If we don't pass a string value during stack build, the default value “t2.micro” will be used.

Accessing the parameter - Now that we have defined the parameter but there should be a way to use that parameter ( if it is passed or default value ). To access the parameter we can write the Resources section as below,

"Ec2Instance" : {
  "Type" : "AWS::EC2::Instance",
  "Properties" : {
    "InstanceType" : { "Ref" : "InstanceType" },
    "ImageId" : "ami-0ff8a91507f77f867"
  }
}

In the above case, we can see that the InstanceType is referring the paramter with the name “InstanceType”. 

5. Mapping are type of logical constructs in cloudformation. They can be taught as a dictionary map. 

Mappings are another logical construct in CloudFormation. Mappings can be thought of as a dictionary map. It’s sort of like a 2-level case statement. The mapping must be exactly 2 levels

"Mappings" : {
  "RegionMap" : {
    "us-east-2a"      : { "32" : "ami-00ed77e0e02f15966", “64”:”ami-00ed77e0e02f15966” },

    "us-west-2b"      : { "32" : "ami-0bdb828fd58c52235", “64” : “ami-0bdb828fd58c52235” }
  
    }
}

These mapping work as a conditional parameter values. It matches a key to the value. The RegionMap mapping matches the regions names to the ami instance. If we use the “us-east-2a” region and use this map, the AMI key will have the value “ami-00ed77e0e02f15966”

Using the mapping - Once we define the mapping, in order to use them we will use the “Fn::FindInMap” function. A Resource section using the above mapping can be written as ,

"Resources" : {
     "sampleEc2Instance" : {
        "Type" : "AWS::EC2::Instance",
        "Properties" : {
           "ImageId" : { 
                 "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "32"]
            },
           "InstanceType" : "m1.small"
        }
     }
 }

In the above snippet, aws will return the ami id based on the region we are in. for example if we are in region us-east-2a and the key passed is 32. So aws will go the mapping and get the value (ami-00ed77e0e02f15966) associated with the key “32” for the region us-east-2a.

5. Conditionals are used to define values based on conditions. For example we can get a value from parameter and decide what to do after that. Lets see a basic example on how conditionals can be used,

Lets define a Parameter with type as below,

"Parameters" : {
    "Type" : {
         "Description" : "Type of the environment",
         "Default" : "test",
         "Type" : "String",
         "AllowedValues" : ["prod", "dev", "test"],
         "ConstraintDescription" : "must specify prod, dev, or test."
    }
  },

Now write a condition statement checking the values passed and setting the type. If the passed value is “prod” ( smaller case), a key “Prod” is defined as below,

"Conditions" : {
    "Prod" : {"Fn::Equals" : [{"Ref" : "Type"}, "prod"]},
    "Dev" : {"Fn::Equals" : [{"Ref" : "Type"}, "dev"]}
    "Test" : {"Fn::Equals" : [{"Ref" : "Type"}, "test"]}
  }

This condition can be used in the Resources section as below. We are checking if the condition “Prod” is set. If the value passed as parameter is “prod”, then the condition “Prod” is set and if this is set, we are defining the InstanceType and ImageId with values.

"Resources:" {
"ProdInstance" : {
     "Type" : "AWS::EC2::Instance",
     "Condition" : "Prod",
     "Properties" : {
        "InstanceType" : "m3.xlarge",
        "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "32" ]}
  }
}

6. Pseudo Parameters are parameters that are predefined by AWS and we can use them in our templates to get values. Values like Availability Zone,Regions, account ID etc are defined as Pseudo parameters. In the above case of mapping , resource section has Pseudo parameter called “AWS::Region” which grabs the current region where we are working. There are other pseudo parameters for our usage.

7. Intrinsic functions are used to pass values that are not available until runtime. One such function is Fn::FindInMap function which gets the value associated with a key that is defined in mapping.

We have seen the below intrinsic function usage in the mapping section and in resource section as below,
“ImageId” : { “Fn::FindInMap” : [ “RegionMap”, { “Ref” : “AWS::Region” }, “32”]},

There are other intrinsic function available for our usage.

5. Resources is the major section in the template. This includes various services that we want to define as a part of our infrastructure. This can be from a Ec2 instance to an Sqs Queue. Each resource that we declare include 3 parts,
A logical name like “myInstance”
A resource Type like “AWS::EC2:Instance” ,“AWS::RDS::DBInstance” etc
Properties for that resources

A simple resource definition will be like,
"Resources" : {
    "simplesamplebucket" : {
       "Type" : "AWS::S3::Bucket"
       }
    }

8. Outputs will return one or more values after the stack is completely executed. For example we may use a stack to launch multiple Ec2 instances and in the output section we can return the ip address associated with the instance that we launched recently.

"Outputs" : {
    "InstanceId" : {
      "Description" : "InstanceId of the newly created Ec2 instance",
      "Value" : { "Ref" : "Instance" }
    }
 }

In the above snippet we are trying to grab the instanceId from the Ec2 instance and referring the Instance Resource defined. Our resource section will have resource defined with name “Instance” and we are outputting the value from that.

Hope this helps in

Read More