Prerequisites for Adding Data Collectors (Amazon Web Services)
1. Identify a server where the Data Collector software will be installed. Server requirements include:
• 64-bit OS. See the APTARE StorageConsole Certified Configurations Guide for supported operating systems.
• Support Java Runtime Environment (JRE) 1.7.
• For performance reasons, APTARE recommends that you do not install Data Collectors on the same server as the StorageConsole Portal. However, if you must have both on the same server, verify that the Portal and Data Collector software do not reside in the same directory.
• Install only one Data Collector on a server (or OS instance).
Prerequisite Amazon Web Services (AWS) Configurations
The Amazon Web Services Data Collector can collect from the following AWS entities:
• S3 Bucket (Details and Usage) - Simple Storage Service (S3) for storage in the cloud
• EC2 Details - Elastic Cloud Compute (EC2) for computing services, much like virtual servers
• Billing Records - Usage and corresponding charges, by service
For additional information about the type of data that is collected, see the Data Collector Policy configuration in
Add an Amazon Web Services (AWS) Policy.
The following steps must be taken in Amazon Web Services (AWS) before a Data Collector can gain read-only access to retrieve data.
The following steps must be taken on the APTARE StorageConsole Portal and Data Collector Servers.
Configure an S3 Bucket to Receive Billing Reports
In Amazon Web Services (AWS), an S3 Bucket (Simple Storage Service Bucket) must be configured to receive billing reports with resources and tags.
1. Create an S3 bucket to collect billing records that will be accessed by the StorageConsole AWS Data Collector.
2. In the AWS Billing and Cost Management Preferences, configure the S3 bucket to Receive Billing Records.
3. Copy the text from the AWS-provided sample policy.
This policy sets the permissions that enable AWS billing to create billing record files in the S3 bucket.
4. In the S3 bucket properties, add a bucket policy by pasting the sample into the policy.
5. Verify the S3 bucket.
Select Cost Allocation Tags
The StorageConsole Amazon Web Services (AWS) Data Collector requires a Detailed Billing Report with Resources and Tags.
2. Once an S3 bucket is verified, select Detailed billing report with resources and tags and save the bucket’s preferences.
This is the only AWS report that is required by the StorageConsole Data Collector.
3. Select Cost Allocation Tags that have been assigned to your AWS resources so that they appear in the billing report and also so that they will be collected by the StorageConsole Data Collector. Tags are user-defined and enable groupings and totals for billing and reporting.
User-defined tags are used for collection of EC2 and S3 resources. These tags are required for cost allocation reporting of the total cost of EC2 instances and S3 buckets.
Note: Amazon Web Services generates a report once or more daily, with additions made daily over the month. Therefore, it may take up to 24 hours until a billing records file appears in the S3 bucket that is being collected by the APTARE StorageConsole Data Collector.
Create an AWS IAM User
Data collection requires an Amazon Web Services (AWS) Identity and Access Management (IAM) user with restricted permissions. This user must have read-only permission to collect billing records from the S3 bucket and also to access the AWS API methods to retrieve data about EC2 resources and any S3 bucket. See also,
Link AWS Accounts for Collection of Consolidated Billing Data.
1. In Amazon Web Services IAM Management Console, create an IAM user, specifically for use by the StorageConsole Data Collector.
a Click Users > Create New Users > enter a user name.
b Ensure that Generate an access key for each user is selected.
This configuration results in the following security credentials: Access Key ID and Secret Access Key.
2. Download the credentials, which you will need later when configuring a Data Collector Policy.
These credentials are required when configuring the StorageConsole AWS Data Collector Policy. The access key and secret access key will be used by the Data Collector to make read-only requests to AWS APIs.
3. In the IAM window, select the IAM User you just created and grant permissions by attaching the AWS-supplied ReadOnlyAccess policy.
This read-only policy allows the Data Collector to retrieve data about EC2 resources and S3 buckets.
Example of a Custom AWS Policy for StorageConsole AWS Collection
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::[Billing Bucket Name]",
"arn:aws:s3:::[Billing Bucket Name]/*"
]
},
{
"Action": [
"ec2:DescribeAccountAttributes",
"ec2:DescribeAddresses",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeHosts",
"ec2:DescribeImages",
"ec2:DescribeInstances",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeRegions",
"ec2:DescribeSnapshots",
"ec2:DescribeSubnets",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DescribeVpcs",
"iam:GetAccountAuthorizationDetails",
"iam:GetUser",
"iam:ListAccountAliases",
"s3:GetBucketLocation",
"s3:GetBucketLifecycleConfiguration",
"s3:GetBucketLoggingConfiguration",
"s3:GetBucketPolicy",
"s3:GetBucketReplicationConfiguration",
"s3:GetBucketTaggingConfiguration",
"s3:GetBucketVersioningConfiguration",
"s3:HeadBucket",
"s3:ListAllMyBuckets",
"s3:ListBucket",
"s3:ListBucketVersions"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
Link AWS Accounts for Collection of Consolidated Billing Data
Some large organizations create separate AWS accounts for different cost centers and then link these accounts to a Payer Account. In this case, the billing records for all of these accounts are accumulated in the billing records for the Payer Account. If you want to have a single AWS IAM user to collect these billing records and all other information about EC2 and S3 buckets, you will need to grant the user cross-account API access. By linking accounts, you establish a trust relationship between the accounts.
Create a Role for StorageConsole Data Collection
1. Log in to the AWS account that is not the Payer Account.
2. In the AWS IAM window, select Roles.
3. Enter a role name that identifies it as the role specifically for data collection, such as readOnlyAccessForCollection. The name you enter cannot be changed once the role is created.
4. Select the Role Type: Role for Cross-Account Access > Provide access between AWS accounts you own.
5. Establish Trust using the Account ID of the Payer Account, but do not require the MFA.
6. Attach the AWS-supplied ReadOnlyAccess policy.
7. Before creating the role, review the role information to ensure that the following information is correct:
• Role Name: Role named specifically for StorageConsole data collection.
• Trusted Entity: ID of the Payer Account.
• Policy: ReadOnlyAccess.
9. Click Create Role to link the accounts.
Add the Role to the IAM User
The role that was created in
Create a Role for StorageConsole Data Collection, must now be added to the IAM User that you created in
Create an AWS IAM User.
1. Log in to the AWS Payer Account.
3. Under the user’s permissions, in addition to the ReadOnlyAccess policy that is listed, create an Inline Policy.
4. Select Custom Policy and enter a Policy Name.
5. Customize permissions by editing the policy, replacing the Resource example (shown in red below) with the Role ARN that you copied to the clipboard, enclosed in straight quotes.
{"Statement":[
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::123456789012:role/accessForAPTARECollector"
}
]
}