Azure Data Lake Storage Gen2 comes out in preview

December 7 2018

Microsoft has revamped and updated the preview of Azure Data Lake Storage (ADLS) Gen2. James Baker, program manager for Azure Storage, shared the slew of new features in a blog post.

Gen2 accounts now support Storage Explorer, increasing options for access control management and data exploration. External tables can be connected in SQL Data Warehouse even when Storage Firewalls are running. Common Data Model support for SQL Data Warehouse and Power BI is being extended in ADLS Gen2, and it will be possible to mount ADLS filesystems into the Databricks File System. The Gen2 preview was first released in June.

For analytics services, Virtual Network and Storage Firewall rules will apply. Similarly, customers will have the option to use keys of their own—or ones provided by Microsoft—for encryption at rest and in transit. Other features will be available soon, such as security features integrated with HDInsight and Databricks. Like ADLS Gen1, Gen2 uses POSIX-compliant Access Control Lists and Role Based Access Control, restricting access to only authorized users and authenticating with Azure Active Directory OAuth 2.0 bearer tokens.

"Once authenticated, permissions are applied at the finest granularity to ensure the right level of authorization for protecting an enterprise's big data assets. End-to-end encryption of data and transport layer protections complete the security shield for an enterprise data lake," Baker stated.

Baker emphasized that as part of the Azure Blobs service, ADLS Gen2 will be available in all Azure regions worldwide. Microsoft views this as an important attribute for many companies that want to run their data lakes close to employees, thus benefiting from lower latency, as they work to comply with mounting sovereignty regulations about where data resides.

"[ADLS Gen2] is the result of converging the capabilities of our two existing storage services, Azure Blob Storage, and Azure Data Lake Storage Gen1. Features from Azure Data Lake Storage Gen1, such as file system semantics, file-level security, and scale are combined with low-cost, tiered storage, and high availability/disaster recovery capabilities from Azure Blob Storage," Kavitha Jonnakuti, principal program manager for Azure SQL Data Warehouse said, in a statement.

The announcement came in parallel with the release of Virtual Network, or VNet Service Endpoints for Azure SQL Data Warehouse, which isolate traffic between a customer's logical server and one or more subnets in their virtual network. VNet Service Endpoints route through Azure's "backbone" network, rather than on-premises or via virtual appliances, at no extra cost to users. The announcement tied in with new PolyBase connectivity to ADLS Gen2 over the Azure Blob File System driver.

FREE Membership Required to View Full Content:

Become a MemberLogin
Joining gives you free, unlimited access to news, analysis, white papers, case studies, product brochures, and more, and it’s all FREE. You’ll also have the option to receive periodic email newsletters with the latest relevant articles and content updates. Learn more about us here
About MSDW Editorial Team

More about MSDW Editorial Team