Menu
Create an Azure Data Lake Storage Container – Data Sources and Ingestion-1

Create an Azure Data Lake Storage Container – Data Sources and Ingestion-1

FIGUER 3.2 An Azure storage account Overview blade Exercise 3.1 walked you through provisioning an ADLS container. You encountered numerous options, beginning with the first items selected, the subscription and resource group. Remember that an Azure subscription is the location where billing happens. It is a grouping of all provisioned Azure products. You can have […]

Where Does Data Come From? – Data Sources and Ingestion-2

Where Does Data Come From? – Data Sources and Ingestion-2

Data also can come from social media outlets in the form of comments or ratings. An Azure Cognitive Service called the Language Understanding Intelligent Service (LUIS) can help you understand the meaning of comments. LUIS converts a comment into a meaning using something called an intent. If the comments contain words such as “bad,” “angry,” […]

Summary – CREATE DATABASE dbName; GO

Summary – CREATE DATABASE dbName; GO

There was a lot covered in this chapter, starting with the description of the first data storage device up to real‐time data analysis and intelligence gathering. You learned about the different file formats, like JSON, CSV, and Parquet. You learned about the different ways in which data can be stored, like structured, semi‐structured, and nonstructured. […]

Training and Enrichment – CREATE DATABASE dbName; GO

Training and Enrichment – CREATE DATABASE dbName; GO

The training and enrichment of data typically happens by making improvements to the data quality or invoking Azure Machine Learning models, which can be later consumed by Azure Cognitive Services. The invoke can take place within a pipeline or manually. Azure Machine Learning models can be used to predict future outcomes based on historical trends […]

Understanding Big Data Processing – CREATE DATABASE dbName; GO

Understanding Big Data Processing – CREATE DATABASE dbName; GO

The previous sections covered much of the mid‐level data theory required to be a great Azure data engineer and give you a good chance of passing the exam. Now it’s time to learn a bit about the processing of Big Data across the various data management stages. You will also learn about some different types […]

SQL Server Integration Services – CREATE DATABASE dbName; GO

SQL Server Integration Services – CREATE DATABASE dbName; GO

Introduced in Chapter 1, SQL Server Integration Services (SSIS) is useful for pulling data from numerous datastores, transforming the data, and storing it in a central datastore for analysis. Ingestion can be initiated by pulling data from existing sources instead, which is in contrast to data producers pushing data into the pipeline. Bulk Copy Program […]

Analytics Types – CREATE DATABASE dbName; GO

Analytics Types – CREATE DATABASE dbName; GO

There are numerous types of data analytics, all of which are supported by Azure products, specifically Azure Synapse Analytics. In most cases, the name of the analytics type is enough to determine its meaning and purpose. However, here are the most common analytics types and a brief summary of each. Descriptive The descriptive analytic type […]

Where Does Data Come From? – Data Sources and Ingestion-1

Where Does Data Come From? – Data Sources and Ingestion-1

Chapter 2, “CREATE DATABASE dbName; GO,” discussed the variety, velocity, and volume characteristics of data. You learned that the velocity and volume of data are increasing exponentially and that those two characteristics are the reason running data analytics in the cloud became necessary. Most companies cannot afford to purchase and maintain the compute and storage […]

Design a Data Storage Structure – Data Sources and Ingestion

Design a Data Storage Structure – Data Sources and Ingestion

In this chapter you will provision numerous Azure data analytics products. By doing so, you will begin to understand more about the products and their features, which can help you create and choose the best tool for your given solution requirements. Choosing a proper service for a scenario results in having a solid design. Table […]

Upload Data to an ADLS Container – Data Sources and Ingestion

Upload Data to an ADLS Container – Data Sources and Ingestion

FIGUER 3.3 The Upload folder in Azure Storage Explorer Refer to Table 3.1 and try to determine what kind of ingestion this was. Did you use the correct processing service for the ingestion type? Yes, you did. What you just performed was an ad hoc upload. One of the recommended tools is noted as Azure […]