Showing posts with label Tips. Show all posts
Showing posts with label Tips. Show all posts

Saturday, July 28, 2018

Azure-Tips & Tricks We Should Know


Tip #01- Enablement Of Ping On Windows VM

Ping is widely used to communicate from one VM to another. It uses the Internet Control Message Protocol (ICMP), which is denied through the Windows Firewall, by default. you can enable the Internet Control Message Protocol (ICMP) through the Windows firewall so that you can ping the second server.Below is the powershell command to achieve he same-

New-NetFirewallRule -DisplayName "Allow ICMPv4-In"-Protocol ICMPv4 
Tip #02- Deny All To Internet 

Most of the times you might hear Architects designing their Security layers by Denying all traffic to the Internet( In case of Azure through NSG) though this will ensure the security of your network from data leakage or malware, but at times it can lead to frustration and can break things which are suppose to function smoothly. Some times your VM's do need access to Azure IP addresses(Service like Azure Storage, Azure SQL Database, etc..).Now as these IPS also falls under the Internet tag, When you are blocking the internet unknowingly you are blocking access to these services as well. There is very good solution for such scenario by Microsoft’s Keith Mayer, you can refer his Article to have more comprehensive approach for this.

Tip #03- Storage Account Key Rotation

Any storage account creation automatically generates the Shared Key(Pair), this shared access key has proven to be a commonly used option to access Aforementioned Storage account. VNET integration with Storage account allows for access to this via VNET Service Endpoints. But this option has its operation radius which at times are limited to IaaS space only. In cases when an Azure Storage Account is accessible over the Internet (PaaS) it will make use of shared access key for accessibility.

Under given situation when an individual possessing these keys goes out of organization, it can leave them vulnerable & exposed.To mitigate this, Azure best practice warrants us to adopt robust key rotation policy, which means rotate them periodically.

When it comes Shared Access Keys they come in Pair, for enterprise grade key rotation solution, we shall integrate the Rotation policy with Azure Automation, Log Analytics and Key Vault.
  • Regenerate Key-2 for Given Storage Account
  • Wait for acceptable time period to ensure Key-2 is active
  • Update the secret in Azure Key Vault (with new value of Key-2)
  • Repeat exact steps for Key-1 Regeneration

Saturday, June 11, 2016

Purge Azure CDN Endpoint

Due to updates to your web application, or to quickly update assets that contain incorrect information.Sometimes you may wish to purge cached content from all edge nodes and force them all to retrieve new updated assets. 
To handle aforementioned scenario the term we use to describe this technology is Purging.Purging particular asset on Azure CDN deletes the cached version of that asset from all Edge servers on the Azure CDN. For example, if you have updated an image, or have made changes to your CSS file. Purging Azure CDN asset prevents an old version of an asset from being delivered to your clients.

Microsoft wasn't supporting the conventional way of purging the content, which intern was really difficult for customer to achieve.Under past procedure you had to contact Microsoft Support and create a ticket to purge the edge servers.Say for an example you have hosted the videos on Azure CDN and because the process was so clumsy with support tickets to force a purge on servers around the world, people would rather uploaded a file with a new name and changed the web page to point to the new file.Or best They will go with Short TTL as an Alternative to Purge.

Using Azure Portal


The support has recently been made available to Azure ARM Portal-
  • In the Azure Portal, browse to the CDN profile containing the endpoint you wish to purge.
  • From the CDN profile blade, click the purge button.


  • On the Purge blade, select the service address you wish to purge from the URL dropdown.


  • Select what assets you wish to purge from the edge nodes. If you wish to clear all assets, click the Purge all checkbox. Otherwise, type the full path of each asset you wish to purge (e.g., /pictures/test.png) in the Path textbox.
  • Click the Purge button.



Using PowerShell

Get-Command -Module AzureRM.Cdn 
To invoke the purge make use of Unpublish-AzureRmCdnEndpointContent
Unpublish-AzureRmCdnEndpointContent `  
    -ResourceGroupName $ResourceGroupName `
    -ProfileName $ProfileName `
    -EndpointName $EndpointName `
    -PurgeContent = '/*'
With endpoint object in hand
$endpoint = Get-AzureRmCdnEndpoint -ResourceGroupName $ResourceGroupName `
    -ResourceGroupName $ResourceGroupName `
    -ProfileName $ProfileName `
    -EndpointName $EndpointName

$endpoint | Unpublish-AzureRmCdnEndpointContent -PurgeContent = '/*'
Related Articles : Solutions In Azure: Azure CDN

Tuesday, June 7, 2016

Best Practices for Creating Azure Resource Manager Templates -Part II

This post is second part of the series : Best Practices for Creating ARM Template, If you haven't already seen please go ahead and check the First part- Best Practices For Creating ARM template

11. Use ARM Template expression function such as trim(), toLower() as a data cleaning measure to ensure input consistency

12. ARM Termination Protection-For Critical Resource always implement Resource Lock

For resources where you have to constrain the ability to commit write actions and protect against accidental deletions, Azure Resource Lock can be a way to go.Check more on how to implement Resource Lock-

{
  "type": "extensions",
  "name": "Microsoft.EnterpriseCloud.Monitoring",
  "apiVersion": "[variables('apiVersion')]",
  "location": "[resourceGroup().location]",
  "dependsOn": [
    "[concat('Microsoft.Compute/virtualMachines/', variables('vmName'))]"
  ],
  "properties": {
    "publisher": "Microsoft.EnterpriseCloud.Monitoring",
    "type": "MicrosoftMonitoringAgent",
    "typeHandlerVersion": "1.0",
    "autoUpgradeMinorVersion": true,
    "settings": {
      "workspaceId": "[parameters('workspaceId')]"
    },
    "protectedSettings": {
      "workspaceKey": "[parameters('workspaceKey')]"
    }
  }
}

13. EnableLog Analytics To Azure VM's- Use OMS

If you are using VMs in Azure you should always opt for some monitoring and analyzing capabilities for them.OMS offers log analytic capabilities that enable you to collect and analyze machine data from virtually any source.Using data generated across devices, operating systems, workloads and user actions, OMS provides instant access to critical information through one integrated console. You can troubleshoot across multiple data sources and easily identify the root cause of operational and security issue.
Log Analytics (OMS) supports a majority of Windows and Linux variations available in Azure today,and because it's been pitched as IT management solution for the hybrid cloud you can manage any instance in any cloud, including on-premises, Azure, AWS, Windows Server, Linux, VMware, and OpenStack

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "lockedResource": {
      "type": "string"
    }
  },
  "resources": [
    {
      "name": "[concat(parameters('lockedResource'), '/Microsoft.Authorization/utLock')]",
      "type": "Microsoft.Storage/storageAccounts/providers/locks",
      "apiVersion": "2015-01-01",
      "properties": {
        "level": "CannotDelete"
      }
    }
  ] } 
The OMS Workspace ID and OMS Workspace Key are required to onboard the OMS Service and can be found in the OMS Portal under Settings > Connected Sources


14. Tag the resources properly

Tagging provides you the ability to include metadata about your resource.Common use cases are to include references to environment types, team or division , individuals accountable.It provides subscription wide taxonomy The benefit of tags is that they can be pulled together in billing roll up or within a summary view.


"tags": { "
   "ProjectName": "[parameters(projectName)]",
   "EnvironmentType" :"[parameters('environmentType')]"
} ,

15. Have the habit for pre-deployment validation

Azure Resource Manager as it stand today has two-part validation: before and during deployment. The before deployment checks whether a template is syntactically correct and that it would be accepted by the Azure Resource Manager. Then the during deployment validation (runtime) checks for resource dependencies within your Azure subscription. The templates may seem valid, but until they are deployed as there is no fool-proof way to guarantee that it is a working template. However, as a good practice when deploying templates, make pre-deployment validation a habit as it will catch most obvious errors detected by Azure Resource Manager.

16. Don't Pass your Passwords as Plain-text:Use Azure key Vault With ARM Template

With template-based deployments that include passwords stored in parameter files (or other insecure stores), there is always the risk of leaking the password should someone access the particular file.

As an It guy we should always took to lock down environments so that only the right groups of users can login and modify the configurations.

Once you have stored your passwords as a Secret in a Key Vault, its ready to be accessed via ARM Template.

The way you can supply Key Vault Secrets to ARM templates is via use of parameters file.

"osProfile": {
    "computername": "[variables('vmName')]",
    "adminUsername": "[parameters('adminUsername')]",
    "adminPassword": "[parameters('adminPassword')]"
}
We can feed in the Secret we pushed to our Key Vault using the below snippet in our parameters file.


"adminPassword": {
     "reference": {
        "keyVault": {
   "id": "/subscriptions/{subscription-guid}/resourceGroups/{keyvault-rg}/providers/Microsoft.KeyVault/vaults/ProvisioningVault"
        },
        "secretName": "LocalAdminPass"
      }
  }

 Separation of Keys from Deployments
A best practice is to maintain separate ARM templates for-
  1. Creation of vaults (which will contain the key material)
  2. Deployment of the VMs (with URI references to the keys contained in the vaults)
17. Include the Output Section
Whenever possible return some output to validate the Deployment.With Outputs section, users can specify values that are returned from deployment.For example,  you could return the URI to access a deployed resource.
"outputs":{
  "siteUri" : {
"type" : "string",
"value": "[concat('http://',reference(resourceId('Microsoft.Web/sites', parameters('siteName'))).hostNames[0])]"
   }
}
18. Use Audit Logs For Deployment Operations

Audit logs would be your first place for troubleshoot because it contains all write operations (PUT, POST, DELETE) performed on your resources.
It also helps you determine who made what changes to the subscription(s) and related Azure resources.In Future these logs can be processed for audit compliance purpose as well.



19. Disable Windows Automatic Updates  

To disable windows updates at provisioning time it self, you have to disable the same under "windowsConfiguration" Setting.This particular section comes under osProfile -

 "osProfile": {
          "computerName": "[variables('vmName')]",
          "adminUsername": "[parameters('adminUsername')]",
          "adminPassword": "[parameters('adminPassword')]",
          "windowsConfiguration": {
            "enableAutomaticUpdates": false
           }
          },

20. Template Formatting 
  • Pass your template through a JSON validator to remove extraneous commas, parenthesis, brackets that may cause an error during deployment. Try JSONlint or a linter package for your favorite editing environment (Visual Studio Code, Atom, Sublime Text, Visual Studio, etc.)
  • Format your JSON for better readability. You can use a JSON formatter package for your local editor. In Visual Studio, format the document with Ctrl+K, Ctrl+D. In VS Code, use Alt+Shift+F. If your local editor doesn't format the document, you can use an online formatter.

ARM TEMPLATES KNOWN LIMITATIONS-


  1. ARM Template file size can't exceed beyond 1 mb 
  2. ARM Parameter Template file Size can't exceed beyong 64 kb 
  3. If parameter name value contains "." character, the arm template deployment will fail in azure portal(bug).EG-My.Database 
  4. Variables doesn't support ARM template expression function such as "reference()"

Related Articles : Best Practice For Creating ARM Template

Wednesday, June 1, 2016

Best Practices for Creating Azure Resource Manager Templates- Part I

Azure Resource Manager (ARM) is the new set of APIs for deploying and managing Microsoft Azure services, From Microsoft point of view they are terming this as revolutionized way of resource management.

AWS has had this capability for quite sometimes, they have named it as CloudFormation. While ARM Templates and CloudFormation are quite identical in their behavior and are trying to achieve similar goals, under the hood there are some basic differences which is beyond the context of this article.

Though Microsoft has got a large repository of ARM Quick Start Templates which is accessible here... yet being a new service overall documentation of ARM is still evolving. At times it's difficult to find samples sufficing your custom requirements, if you are caught into that space there are few tips which definitely would help you.Under this article i will outline few important considerations while authoring your own ARM Template-

1. Best Authoring Tool For ARM Template :Visual Studio 2015


The ability to author and manage ARM Templates within Visual Studio 2015 is contained within the “Azure Resource Group” project template. This integration of ARM template and VS2015 has given developers a sense of belonging towards solution-oriented templates. Here with this they can visualize the resources and parameters using JSON outline. With Visual Studio you can add resources easily, make great inbuilt IntelliSense and validation of Visual Studio JSON editor. It also offers support for wizard based deployment, as part of best practices you can integrate your artifacts and deployment templates with source control.

2. A tool for assistance:Azure Resource Explorer


This is an essential tool for anybody writing ARM templates. It can be a great way to understand the resource hierarchy and find where to put configuration. You can deploy a resource using the portal and use the resource explorer to see the JSON schema for the resource you just created you can make a change in the Portal and then find the corresponding entries in Resource Explorer and include it to your template! That way it can help you figure out the settings for your templates. https://resources.azure.com/

3. Version your Azure Resource Manager Templates


ContentVersion is a required element of ARM Template structure. This specifies Version of the template (such as 1.0.0.0). You can provide any value for this element. When deploying resources using the template, this value can be used to make sure that the right template is being used. Another benefit is it makes template artifact-able with version control.
{
  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
   "contentVersion": "",
   "parameters": {  },
   "variables": {  },
   "resources": [  ],
   "outputs": {  }
}
4. Use Latest version of the resource provider API

{
   "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
   "contentVersion": "",
   "parameters": {  },
   "variables": {  },
   "resources": [  ],
   "outputs": {  }
}

Location of the JSON schema file that describes the version of the template language. This will pull in the proper version of all the child schemas. to get the latest Api Version for the resource providers you’re going to use keep checking this 
Always try to use the updated Schema. But in general you should be able to stick with the top level schema.
5. Use Input Parameters

Under this section specify values you can input while deploying the resources. These parameter values enable you to customize the deployment by providing values that are tailored for a particular environment, Site Name , Region etc.. To our benefit this makes the While template configurable and maintainable.

"parameters": {
   "siteName": {
      "type": "string",
      "minLength": 2,
      "maxLength": 60
   },
   "siteLocation": {
      "type": "string",
      "minLength": 2
   },
   "hostingPlanName": {
      "type": "string"
   },  
   "hostingPlanSku": {
      "type": "string",
      "allowedValues": [
        "Free",
        "Shared",
        "Basic",
        "Standard",
        "Premium"
      ],
      "defaultValue": "Free"
   },
   "instancesCount": {
      "type": "int",
      "maxValue": 10
   }
   
}
For region Mapping always go with Drop-down,that can be achieved by specifying values in allowed value section..   
6. Work With Variables

Variables in ARM template can be used to simplify the template.Under this section we construct values that can be used throughout our template. Cases where same thing (ie Connection String) has to used multiple places it's not worthy repeating the same string again & again.Any sort of human errors will result in failure during the deployment. Instead, we can assign these things as a value to a variable and refer that variable within the template as needed. Also, variables in ARM template language will let us expand or construct other variable values.

"variables": {
   "connectionString": "[concat('Name=', parameters('username'), ';Password=', parameters('password'))]"
}

7. Decompose The Larger Deployment– Using Linked Templates

As part of ARM Template best practice considerations It's recommended to separate your template into multiple templates,if applicable, into multiple target-specific templates. Think of this as creating re-usable code. You can leverage the re-usable parts of your code within multiple aspects of your application or the deployment.

Template linking enables you to link to and execute another template,It enables you to decompose your deployment into a set of targeted, purpose-specific templates. Just as with decomposing an application into a number of code classes, decomposition provides benefits in terms of testing, re-use, and readability.


You create a link between two templates by adding a deployment resource within the main template that points to the linked template. You set the templateLink property to the URI of the linked template.

"resources": [ 
  { 
     "apiVersion": "2015-01-01", 
     "name": "nestedTemplate", 
     "type": "Microsoft.Resources/deployments", 
     "properties": { 
       "mode": "incremental", 
       "templateLink": {
          "uri": "https://www.contoso.com/AzureTemplates/newStorageAccount.json",
          "contentVersion": "1.0.0.0"
       }, 
       "parameters": { 
          "StorageAccountName":{"value": "[parameters('StorageAccountName')]"} 
       } 
     } 
  } 
] 

It's also takes care of any limit imposed by Microsoft from Template size prospective...
8. Deploying multiple instances of resources: Use copy Object

you’ll want to deploy multiple instances of a given resource,For example- multiple web servers or your Hadoop cluster might have multiple data nodes.Within the resource to create the same multiple times, you can define a copy object that specifies the number of times to iterate.
You can use the copyIndex() function in a template to provide details about the current index of the loop being executed(Current Iteration).

"parameters": { 
  "count": { 
    "type": "int", 
    "defaultValue": 3 
  } 
}, 
"resources": [ 
  { 
      "name": "[concat('examplecopy-', copyIndex())]", 
      "type": "Microsoft.Web/sites", 
      "location": "East US", 
      "apiVersion": "2015-08-01",
      "copy": { 
         "name": "websitescopy", 
         "count": "[parameters('count')]" 
      }, 
      "properties": {
          "serverFarmId": "hostingPlanName"
      }
  } 
]
9. Defining Resource Dependencies:Use DependsOn

For a given resource, there can be other resources that must exist before the resource is deployed. For an Example- Say for a given VM, you may be dependent on having a database resource successfully provisioned.You define this relationship by marking one resource as dependent on the other resource. You can define dependencies on other resources using dependsOn and resources property of a resource. A dependency can also be specified using the reference function.

{
  "type": "Microsoft.Compute/virtualMachineScaleSets",
  "name": "[variables('namingInfix')]",
  "location": "[variables('location')]",
  "apiVersion": "2016-03-30",
  "tags": {
    "displayName": "VMScaleSet"
  },
  "dependsOn": [
    "storageLoop",
    "[concat('Microsoft.Network/loadBalancers/', variables('loadBalancerName'))]",
    "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
  ],
  ...
}

Above example shows a virtual machine scale set that is dependent on a load balancer, virtual network, and a loop that creates multiple storage accounts.


10. Define Api Versions for different Resource Providers as variables instead of parameters, Much easier to find and replace.

Related Articles : Best Practice For Creating ARM Template

Wednesday, May 11, 2016

Solutions in Azure : Azure CDN for Maximum Bandwidth & Reduced Latency - Part II

From First Part of the Article we can derive a conclusion that the CDN’s job is to enhance your regular hosting by reducing bandwidth consumption, minimizing latency and providing the scalability needed to handle abnormal traffic loads. It cuts down on round-trip time (RTT), effectively giving similar response to end user irrespective of their geographical presence.

In MicroMarketMonitor’s recent report, it has clearly mentioned that only the North American content delivery network market is expected to grow from $1.95 billion in 2013 to $7.83 billion in 2019. One significant factor driving this growth is end user interaction with online content. So moving forward its gona be a major factor while architecting any only application--

Azure CDN Highlights


1. Improve rendering speed & Handle high traffic loads
Azure CDN servers the content by making use of its large network of POPs. This dramatically increases the speed and availability, resulting in significant user experience improvements.
2. Designed for today’s web
Azure CDN is specifically designed for the dynamic, media-centric web of today—and cater to the requirement of its users who expect everything to be fast, high quality, and always-on. 
3. Streaming Aware Era
Azure CDN can be helpful under all possible three ways while serving videos over HTTP- Progressive Download and Play, HTTP Pseudostreaming & Live Streaming. 
4. Dynamic Content Acceleration
If you understand the nitty-gritties of Azure CDN it also uses series of techniques to serve uncatchable content faster. For example, it can route all communication from a client in India to a server in the US through an edge in India and an edge in the US. They then maintain a constant connection between those two edges and apply WAN optimization techniques to accelerate it. 
5. Block spammers, scrapers and other bad bots
Azure Content Delivery Network is built on a highly scalable, reverse-proxy architecture with sophisticated DDoS identification and mitigation technologies to protect your website from DDoS attacks. 
6. When the expectations are at peak, Azure CDN delivers
Thanks to its distributed global scale, Azure Content Delivery Network handles sudden traffic spikes and heavy loads, like the start of a major product launch or global sporting event.

Working With Azure Storage


Once the CDN is enabled on an Azure storage account, any blobs that are in public containers and are available for anonymous access will be cached via the CDN. Only blobs that are publicly available can be cached with the Azure CDN. To make a blob publicly available for anonymous access, you must denote its container as public. Once you do so, all blobs within that container will be available for anonymous read access. You have the option of making container data public as well, or restricting access only to the blobs within it.

For best performance, use CDN edge caching for delivering blobs less than 10 GB in size.
When you enable CDN access for a storage account, the Management Portal provides you with a CDN domain name in the following format: http://.vo.msecnd.net/. This domain name can be used to access blobs in a public container. For example, given a public container named music in a storage account named myaccount, users can access the blobs in that container using either of the following two URLs:
  • Azure Blob service URL: http://myAccount.blob.core.windows.net/music/
  • Azure CDN URL: http://.vo.msecnd.net/music/

Working With Azure Websites


You can enable CDN from your websites to cache your web contents, such as images, scripts, and stylesheets. See Integrate an Azure Website with Azure CDN.
When you enable CDN access for a website, the Management Portal provides you with a CDN domain name in the following format: http://.vo.msecnd.net/. This domain name can be used to retrieve objects from a website. For example, given a public container named cdn and an image file called music.png, users can access the object using either of the following two URLs:

  •  Azure Website URL: http://mySiteName.azurewebsites.net/cdn/music.png
  • Azure CDN URL: http://.vo.msecnd.net/cdn/music.png

Working With Azure Cloud Services


You can cache objects to the CDN that are provided by an Azure cloud service. Caching for cloud services has the following constraints:
  • The CDN should be used to cache static content only.
  • Your cloud service must be deployed to in a production deployment.
  • Your cloud service must provide the object on port 80 using HTTP.
  • The cloud service must place the content to be cached in, or delivered from, the /cdn folder on the cloud service
When you enable CDN access for a cloud service, the Management Portal provides you with a CDN domain name in the following format: http://.vo.msecnd.net/. This domain name can be used to retrieve objects from a cloud service. For example, given a cloud service named myHostedService and an ASP.NET web page called music.aspx that delivers content, users can access the object using either of the following two URLs:
  •  Azure cloud service URL: http://myHostedService.cloudapp.net/cdn/music.aspx 
  • Azure CDN URL: http://.vo.msecnd.net/music.aspx

Accessing Cached Content over HTTPS

Azure allows you to retrieve content from the CDN using HTTPS calls. This allows you to incorporate content cached in the CDN into secure web pages without receiving warnings about mixed security content types.

To serve your CDN assets over HTTPS there are couple of constraints worth mentioning:

  • You must use the certificate provided by the CDN. Third party certificates are not supported.
  • You must use the CDN domain to access content. HTTPS support is not available for custom domain names (CNAMEs) since the CDN does not support custom certificates at this time.

Even when HTTPS is enabled, content from the CDN can be retrieved using both HTTP and HTTPS.

Note: If you've created a CDN for an Azure Cloud Service (e.g. http://[XYZ].cloudapp.net/cdn/) it's important that you create a self-signed certificate for your Azure domain ([XYZ].cloudapp.net). If you're using Azure Virtual Machines can be done through IIS.

Custom Domain to Content Delivery Network (CDN) endpoint

In case you want to access the cached content with custom domain, azure lets you map your domain to particular CDN End point. With that in place you can use your own domain name in URLs to retrieve the cached content.

For detailed information on implementation please check- Map CDN to Custom Domain

CDNs are an essential part of current generation’s Internet, and they’re only going to become more important as time goes on. Even now, companies are hard at work trying to figure out ways to move more functionality to edge servers (POP Locations) in order to provide users with the fastest possible experience. Azure CDN plays a vital role as it suffice current generation CDN requirement. While implementing Azure CDN (Or any CDN for that matter) the important thing is to formulate a strategy regarding the maximum lifespan of an object beforehand.

P.S.: As of Now you can't use your own custom domain or SSL Certificate with Azure CDN, But that support will eventually be coming.

Related Articles : Solutions In Azure: Azure CDN