Everything You Need to Know to Migrate Your Data and Analytics Workloads to the Cloud

Discuss Your Cloud Strategy

In this blog, we’ll walk through our street proven approach to migrating analytic workloads to the cloud.

One of the biggest benefits of the cloud is that the barrier to entry is low because it’s very easy to spin up a server and start playing around.​ ​We recommend that you balance the ease with proper planning so that you can:

  • Ensure that your organization is prepared from a financial, process, and governance perspective and bought-in from a cultural perspective;
  • Validate that technology and license agreements support a cloud model​;
  • Define an execution, testing, and communication plan to minimize the impact to business users​; and
  • Properly leverage the rapid pace of innovation and agility that cloud offers over on-prem though continuously optimizing workloads.

Assessment

As with most analytics projects of any magnitude, we recommend that you start with a cloud readiness assessment ​to determine if you’re ready to make the move by reviewing your current environment – corporate and remote workforces, business needs, and priorities – your data, regulatory and security concerns, existing on-premises and cloud infrastructure, applications, and dependencies. ​With a firm understanding of your operations and existing assets, you’ll have a better idea of the best path forward for your cloud migration.​

The steps of a cloud readiness assessment should include:

  • One-on-one information gathering: These sessions should be held with key IT and business stakeholders at your organization​ to identify shared goals and areas of misalignment among stakeholders. There should be a common understanding of what a cloud migration means to the business and the expected results at project completion.​
  • Identify goals and opportunities: Is your goal to reduce cost through lower capital commitments or decommissioning data centers?​ Increase cost transparency within an ITIL framework where you are passing IT costs back to the business?​ Increase performance by leveraging autoscaling, self-healing, and rapid hardware changes​? Increase agility with the ability to try new things and fail/succeed faster​? Whatever your goals are, document them up front and calculate the ROI to help prioritize the objectives with the greatest potential business benefit.
  • Determine cultural readiness​: Often overlooked, this step is important because key stakeholders should be prepared for the shift in mindset that cloud brings. Is your organization prepared to address the technical debt and move away from legacy systems? Are you open to a cloud-native approach where the greatest benefit comes from agility and not just a lift-and-shift to new data center?​ Ultimately, if your key stakeholders are thinking of a cloud migration as simply a move to a new “cheaper” data center, they’re missing the point and the opportunity.​
  • Determine what will be migrated: Which applications, processes, and infrastructure will be migrated. This is especially relevant for data and analytics workloads with consideration to networking latency and data volumes. Licensing stipulations should also be considered at this step.
  • Calculate current TCO​: Total cost is more than just server costs. There are lots of other factors including physical data center property, cooling, electricity, physical security, disaster recover, hardware and licensing, and data center staff. While cloud represents an opportunity for cost savings, if you’re not planning to decommission current resources, then you will be adding to your costs.

While cloud represents an opportunity for cost savings, if you’re not planning to decommission current resources, then you will be adding to your costs.

Plan

When it’s clear where you’re coming from, where you’re going, and who’s going to support you along the way, the next step is to develop a detailed plan. The cloud migration plan should contain the following elements.

Workload Assessment and Prioritization: Determine what can and cannot be migrated to the cloud.​ Many will start with a database platform migration which can be a “lift-and-shift” (moving an application or operation from one environment to another without stopping to redesign the app or operations workflow) or a migration from a legacy database to a cloud-native database. ​We’ve seen an interesting trend lately in that more companies are open to embracing open-source architectures and specifically open-source database engines like Postgres or MySQL to replace proprietary commercial offerings like Oracle and SQL Server and reduce costs.

From there, determine the mechanism of data ingestion. Will it come from an on-premises system or maybe a SaaS platform?​ This all factors into how much you need to invest in your network. Where you source your data from and what platform you’re pushing it into will guide your decision to leverage an existing on-prem ETL/ELT framework or migrate your data transformations to cloud-native or cloud-enabled tools or frameworks.​

Even if you’re not moving your database or other source systems to your cloud environment, analytics platforms are another common first move into the cloud. In many cases you can start with a lift-and-shift to an IaaS provider. In other cases, you may consider a re-platform to SaaS offerings from the same vendor.​

Don’t neglect embedded analytics or other integrations—such as dashboards in salesforce, SharePoint, mashup frameworks, or other third-party tools—or your authentication mechanism.

Platform Selection: Determine the appropriate platforms for your migration. It’s possible that different cloud models will be employed for different parts of the same overall workload.​ SaaS platforms, such as Power BI, Salesforce.com, Qlik, and Tableu, are turnkey offerings that provide you with an application as a service. You are completely insulated from and unburdened by the entirety of the tech stack behind the scenes.​ PaaS offerings, such as Snowflake, allow greater control and require additional management.​ IaaS offerings, such as AWS and Azure, are best equipped to support a broad variety of analytics workflows, but they also require a greater investment in the planning phases and the continued operational management. And while traditionally thought of as IaaS providers, AWS and Azure also have PaaS offerings (like Microsoft Azure SQL and Amazon RDS) and SaaS offerings (like Amazon QuickSite and PowerBI​).

It’s not uncommon to leverage more than one of these options in a comprehensive cloud-based modern data architecture.

It’s not uncommon to leverage IaaS, PaaS, and SaaS platform options in a comprehensive cloud-based modern data architecture.

Macro Execution Plan – Big Bang, Phased, or Hybrid: With a big-bang approach, the project can be completed faster, but also with greater risk.​ A phased approach usually takes more time, but the migration team can focus on smaller chunks of the workload during each phase.​ If going this route, consider breaking it out by business unit or end-user application, and break up the complexity into separate phases to reduce risk in each phase.​ The hybrid approach often entails deploying the architecture to the cloud, but migrating content in phases, often aligned with business units or lines of business. Like a phased approach, you’ll probably run your legacy stack and cloud deployment in parallel until you can comfortably cut-over.

Micro Executions Plan: This is where you consider the individual needs of each tool or workload to determine the best micro-execution plan. A common industry term for these micro-execution approaches is the “6R”.

First, consider what is to be retired or retained as it currently exists. This enables you to immediately trim down your area of focus. The other four Rs of the “6R” cover the most common migration paths:

  • Re-hosting is often called a “lift-and-shift” of the applications. It’s generally the simplest approach but limits cloud-native opportunities.
  • Re-platforming means that the underlying architecture stays the same between cloud and on-prem, but the systems that run on the infrastructure change.
  • Replace can involve moving to a different product with a new architecture or the same product with a new service delivery model, like IaaS to SaaS.
  • Re-architecting involves redesigning the application and infrastructure architecture using cloud-native and often (but not always) open-source tools, technologies, and frameworks like Serverless, Kubernetes and containerization, cloud native or serverless databases, streaming data ingestion, and Amazon EC2 Spot instances.

Foundational Architecture:

  • Networking: Start with networking requirements so that you establish a resilient and performant foundation upon which your cloud solution can be built. If you’re all-in on the cloud, you may not need anything beyond a direct VPN to manage the platform.​ If you host in your facility and data volumes are reasonable, a site-to-site VPN may be adequate.​ When dealing with large data volumes coming from on-prem systems, AWS Direct Connect or Azure Express Route enable low latency connectivity directly into the cloud provider’s backbone.​
  • Identity & Access Management: If your application requires domain membership, you can connect to your on-prem domain controller or deploy a new one to the cloud environment. ​If using Windows authentication for your application, domain connectivity might be required; however, SAML authentication could be an option to avoid a domain​
  • Security: Security has long been a concern with cloud environments, but the native tooling and push-button options at your disposal give you incredible power and flexibility to secure your environment. Defined user roles and responsibilities and a proper change management framework should be implemented to avoid the human error responsible for most system breaches. ​A best practice is to deploy encryption in transit and at-rest across your cloud landing zone; it’s simple to deploy and maintain in modern cloud environments. ​
  • Data Protection Framework: Data Protection is an extension of security. Snapshots and backups can be scheduled and retained or rotated as needed (but remember to monitor the costs associated with storage). Disaster recovery and resiliency are robust and attractive features in modern cloud platforms. Rather than manage dedicated disaster recovery sites or co-location facilities, there are other scenarios like hot/cold, pilot light, or complete load-balanced redundancy across geographic regions to ensure business continuity.​ Consider the revenue impact of a system going down before you define your ​disaster recovery strategy.
  • Automation: Consider investing in Infrastructure as Code (IaC ) from the outset.​ Native tooling or third-party offerings allow you to define your infrastructure in templated and parameteriz​ed configuration files so you can quickly re-deploy you stack.

Execute

Execute Stage of a Cloud Migration

This diagram represents the general steps but will vary depending upon your migration strategy and architecture.​

  1. Start by deploying the foundational framework: this includes the actual cloud provider and setting up identity access management, networking, security, tagging and cost management, and automation.
  2. Migrate data: start with one database, take a phased approach, or do it all at once.
  3. Migrate applications: establish connectivity, make sure everything is configured properly, ensure scheduled jobs are running, etc.
  4. Validate and document: ensure accuracy and reliability
  5. Initial optimization/adaptation: there will always be some minor things to adjust to adapt to the cloud; do initial optimizations
  6. Cutover: This will happen many times if you are taking a phased approach

Operate and Optimize

Once in production, keep optimizing your environment. ​The following are not one-time tasks; they should be performed continuously.

      • Cost Optimization: Continuous right-sizing enables you to start with what you need now and grow with your business demand rather than estimating a multi-year growth plan.​ Using reserved or spot capacity can reduce your infrastructure costs by 20-50% depending upon your level of commitment​. Use cost attribution and budget alerts to track over-run and budget accuracy. ​
      • Process Automation​: Patching can be automated through native tooling and reduce the load on operational teams.​ Test backups and disaster recovery occasionally to ensure your plan is still effective.​ Self-healing and Autoscaling help meet seasonal business demand without a year-round commitment​. Automate shutdown of Dev/QA resources – doing this on weekends alone can immediately save you 30% with on-demand pricing. Shutting down overnight and on weekends can save up to 70%.
      • Improve Operational Efficiency​: Set up hardware monitoring/alerting to proactively investigate problems​. Data transfer can be costly if not properly planned and monitored, but there are numerous options to drastically reduce this spend.​ Also, you’ll often find operational or cost efficiencies by keeping up with new product and feature releases within your platform.

Automate shutdown of Dev/QA resources – doing this on weekends alone can immediately save you 30% with on-demand pricing. Shutting down overnight and on weekends can save up to 70%.


While the above steps are a great guide for your move to the cloud, remember that no migration will look the same. Your cloud migration process and overall strategy should be based on a long term direction that delivers real business value. Take the time to think about how cloud can be utilized to promote success at your organization.

Questions about how cloud can benefit your company?

We offer a free, one-hour cloud strategy session where you’ll get practical advice that you can use to get started or optimize your existing data and analytics workloads in the cloud.

Chance Barkley is a Dallas-based Managing Consultant and the AWS Practice Lead at Analytics8. He specializes in building, deploying, and maintaining cloud-based modern data architectures spanning best of breed tools and technologies.

 

Contact Us

National Office Telephone | Mon-Fri 8:30am-5:30pm CT

Data Strategy Session

To thrive with your data, your people, processes, and technology must all be data-focused. This may sound daunting, but we can help you get there. Sign up to meet with one of our analytics experts who will review your data struggles and help map out steps to achieve data-driven decision making.

Learn More →

Fill out this form to get a 30-minute Data Strategy Session with one of our analytics experts.

Cloud Strategy Session

In one hour, get practical advice that you can use to initiate or continue your move of data and analytics workloads to the cloud.

During your free one-hour cloud strategy session, we will:

  • Review how the cloud fits into overall corporate strategy
  • Review how the cloud fits into data and analytics strategy
  • Review existing cloud assets
  • Discuss data and current analytics solutions to prioritize what components should be moved to the cloud

Fill out this form to get a one hour Cloud Strategy Session with one of our analytics experts.