AWS CLI - Configuring the Command Line Interface for Aws

AWS CLI is an open source tool from Aws that enables Aws platform users to communicate and issues commands on various services of the Aws. This tool lets users to issue commands which in turn invoke the aws services via APIs exposed by those services. The authentication and authorization of the user issuing the commands are abstracted by configuring the Cli with the users credentials or keys. 

The Cli provides higher level of operations on the services as simple services to the users. By having this Cli, users can now create shell scripts or automate simple and complex tasks on the services.

Installation - Installing Cli is quite easy. Though there are few prerequisites like having a python boto library. We can install the Cli using python pip as “pip install awscli”.

Configuration - Once we have the tool available we can run the “aws” command as below,

jagadishm@[/Volumes/Work]: aws

usage: aws [options] [ ...] [parameters]

To see help text, you can run:

  aws help

  aws help

  aws help

Configure the access_key_id and secret_access_key from IAM -> Access keys -> Generate. This will create a keyId and an access key. 

Once a Key id and access key are generated, we will now configure the aws cli as,

jagadishm@[/Volumes/Work]: aws configure

AWS Access Key ID [None]: AKIAIHJJ6T4NSENA4M3A

AWS Secret Access Key [None]: **********

Default region name [None]: us-east-2a

Default output format [None]: 

Configuration files

Once we configure the aws cli, 2 files will be created in the ~/.aws directory.

~/.aws/config - This contains configuration of multiple named profiles including the default profile. The configuration of each profile consist the key/value parts for a region and its output properties.

jagadishm@[~/.aws]: cat config 

[default]

region = us-east-2a

output = json

~/.aws/credentials - This file contains the credentials of multiple named profiles including the default one. The credentials are available for each profile that we have in the config file.

jagadishm@[~/.aws]: cat credentials 

[default]

aws_access_key_id = AKIAIHJJ6T4NSENA4M3A

aws_secret_access_key = *****************

Named Profile - Aws cli supports using named profiles. The named profiles are nothing but profile which has a specific name and aws cli can use this name when invoking services in aws. A named profile can be created as,

jagadishm@[~/.aws]: aws configure --profile admin

AWS Access Key ID [None]: AKIAIHJJ6T4NSENA4M3A

AWS Secret Access Key [None]: *************

Default region name [None]: us-east-2a

Default output format [None]: json

Now if we go to the ~/.aws/config, we can see a new profile with the name created as below,

jagadishm@[~/.aws]: cat config 

[default]

region = us-east-2a

[profile admin]

output = json

region = us-east-2a

List Config Data - Aws cli allows us to list the data available using,

jagadishm@[~/.aws]: aws configure list

      Name                    Value   Type Location

      ----                    -----       ---- --------

   profile                      None None

access_key     ****************4M3A     shared-credentials-file    

secret_key     ****************PPIr        shared-credentials-file    

    region               us-east-2a         config-file ~/.aws/config

jagadishm@[~/.aws]: aws configure list --profile admin

      Name                    Value Type   Location

      ----                    ----- ----   --------

   profile                    admin manual --profile

access_key     ****************4M3A shared-credentials-file    

secret_key     ****************PPIr shared-credentials-file    

    region               us-east-2a config-file    ~/.aws/config

Try out basic commands - Once that we have configured the aws cli, we can run a few commands as below,

List all s3 buckets in the current region of us-east-2a

jagadishm@[~/.aws]: aws s3 ls

2019-07-03 11:23:13 cf-templates-1kdx3hvziz90i-us-east-1

2019-07-03 11:26:35 sample-bucket-simplesamplebucket-1tuihgl33nnsy

Using Named profile

jagadishm@[~/.aws]: aws s3 ls --profile admin

2019-07-03 11:23:13 cf-templates-1kdx3hvziz90i-us-east-1

2019-07-03 11:26:35 sample-bucket-simplesamplebucket-1tuihgl33nnsy

Hope this helps in getting you started with Aws Cli.

Read More

Cloud Formation - Understanding the Template

In this second article of Cloud formation, we will understand some more details of the template and various sections that can be used in the template.

We already know that the definition of the infrastructure is done inside a template file. The template is a JSON format file that describes our infra. This template has to adhere the AWSTemplateFormat guidelines.

1. The AWSTemplateFormatVersionsection (optional) identifies the capabilities of the template.

  {

    “AWSTemplateFormatVersion”: “2010-09-09”

  }

2. Description allows to write comments for the template file that we are creating.

 {

    “AWSTemplateFormatVersion” : “2010-09-09”,

     “Description” : “example template” 

  }

3. Metadata allows us to include more details about the template. Additional details for the resources that we create will be added in this optional metadata section.

"Metadata" : {

  "Instances" : {

           "Description" : "this is for app servers"

    }

}

4. Parameter allows us to customise the template by passing values as inputs when we are creating the stack. This way we can pass different values each time when building the stacks. A parameter can be defined as,

"InstanceType" : {

      "Type" : "String",

      "Default" : "t2.micro",

      "Description" : "Type of EC2 instance to launch"

}

"KeyName" : {

     "Description" : "EC2 Key Pair for SSH Access

     "Default" : "sample",

     "MinLength": "1",

     "MaxLength": "64",

     "AllowedPattern" : "[-_ a-zA-Z0-9]*",

}

In the above snippet we are defined a parameter called “InstanceType”. The parameter type is the string which means we will be passing a string value as input when building the stack. The parameter supports multiple properties like providing default values, length, max and min values etc.

The above InstanceType parameter is string type and default value it t2.micro. If we don't pass a string value during stack build, the default value “t2.micro” will be used.

Accessing the parameter - Now that we have defined the parameter but there should be a way to use that parameter ( if it is passed or default value ). To access the parameter we can write the Resources section as below,

"Ec2Instance" : {

  "Type" : "AWS::EC2::Instance",

  "Properties" : {

    "InstanceType" : { "Ref" : "InstanceType" },

    "ImageId" : "ami-0ff8a91507f77f867"

  }

}

In the above case, we can see that the InstanceType is referring the paramter with the name “InstanceType”. 

5. Mapping are type of logical constructs in cloudformation. They can be taught as a dictionary map. 

Mappings are another logical construct in CloudFormation. Mappings can be thought of as a dictionary map. It’s sort of like a 2-level case statement. The mapping must be exactly 2 levels

"Mappings" : {

  "RegionMap" : {

    "us-east-2a"      : { "32" : "ami-00ed77e0e02f15966", “64”:”ami-00ed77e0e02f15966” },

    "us-west-2b"      : { "32" : "ami-0bdb828fd58c52235", “64” : “ami-0bdb828fd58c52235” }

  

    }

}

These mapping work as a conditional parameter values. It matches a key to the value. The RegionMap mapping matches the regions names to the ami instance. If we use the “us-east-2a” region and use this map, the AMI key will have the value “ami-00ed77e0e02f15966”

Using the mapping - Once we define the mapping, in order to use them we will use the “Fn::FindInMap” function. A Resource section using the above mapping can be written as ,

"Resources" : {

     "sampleEc2Instance" : {

        "Type" : "AWS::EC2::Instance",

        "Properties" : {

           "ImageId" : { 

                 "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "32"]

            },

           "InstanceType" : "m1.small"

        }

     }

 }

In the above snippet, aws will return the ami id based on the region we are in. for example if we are in region us-east-2a and the key passed is 32. So aws will go the mapping and get the value (ami-00ed77e0e02f15966) associated with the key “32” for the region us-east-2a.

5. Conditionals are used to define values based on conditions. For example we can get a value from parameter and decide what to do after that. Lets see a basic example on how conditionals can be used,

Lets define a Parameter with type as below,

"Parameters" : {

    "Type" : {

         "Description" : "Type of the environment",

         "Default" : "test",

         "Type" : "String",

         "AllowedValues" : ["prod", "dev", "test"],

         "ConstraintDescription" : "must specify prod, dev, or test."

    }

  },

Now write a condition statement checking the values passed and setting the type. If the passed value is “prod” ( smaller case), a key “Prod” is defined as below,

"Conditions" : {

    "Prod" : {"Fn::Equals" : [{"Ref" : "Type"}, "prod"]},

    "Dev" : {"Fn::Equals" : [{"Ref" : "Type"}, "dev"]}

    "Test" : {"Fn::Equals" : [{"Ref" : "Type"}, "test"]}

  }

This condition can be used in the Resources section as below. We are checking if the condition “Prod” is set. If the value passed as parameter is “prod”, then the condition “Prod” is set and if this is set, we are defining the InstanceType and ImageId with values.

"Resources:" {

"ProdInstance" : {

     "Type" : "AWS::EC2::Instance",

     "Condition" : "Prod",

     "Properties" : {

        "InstanceType" : "m3.xlarge",

        "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "32" ]}

  }

}

6. Pseudo Parameters are parameters that are predefined by AWS and we can use them in our templates to get values. Values like Availability Zone,Regions, account ID etc are defined as Pseudo parameters. In the above case of mapping , resource section has Pseudo parameter called “AWS::Region” which grabs the current region where we are working. There are other pseudo parameters for our usage.

7. Intrinsic functions are used to pass values that are not available until runtime. One such function is Fn::FindInMap function which gets the value associated with a key that is defined in mapping.

We have seen the below intrinsic function usage in the mapping section and in resource section as below,

“ImageId” : { “Fn::FindInMap” : [ “RegionMap”, { “Ref” : “AWS::Region” }, “32”]},

There are other intrinsic function available for our usage.

5. Resources is the major section in the template. This includes various services that we want to define as a part of our infrastructure. This can be from a Ec2 instance to an Sqs Queue. Each resource that we declare include 3 parts,

A logical name like “myInstance”

A resource Type like “AWS::EC2:Instance” ,“AWS::RDS::DBInstance” etc

Properties for that resources

A simple resource definition will be like,

"Resources" : {

    "simplesamplebucket" : {

       "Type" : "AWS::S3::Bucket"

       }

    }

8. Outputs will return one or more values after the stack is completely executed. For example we may use a stack to launch multiple Ec2 instances and in the output section we can return the ip address associated with the instance that we launched recently.

"Outputs" : {

    "InstanceId" : {

      "Description" : "InstanceId of the newly created Ec2 instance",

      "Value" : { "Ref" : "Instance" }

    }

 }

In the above snippet we are trying to grab the instanceId from the Ec2 instance and referring the Instance Resource defined. Our resource section will have resource defined with name “Instance” and we are outputting the value from that.

Hope this helps in

Read More