Azure Data Lake Storage Gen2 comes out in preview

Microsoft has revamped and updated the preview of Azure Data Lake Storage (ADLS) Gen2. James Baker, program manager for Azure Storage, shared the slew of new features in a blog post.

Gen2 accounts now support Storage Explorer, increasing options for access control management and data exploration. External tables can be connected in SQL Data Warehouse even when Storage Firewalls are running. Common Data Model support for SQL Data Warehouse and Power BI is being extended in ADLS Gen2, and it will be possible to mount ADLS filesystems into the Databricks File System. The Gen2 preview was first released in June.

For analytics services, Virtual Network and Storage Firewall rules will apply. Similarly, customers will have the option to use keys of their own—or ones provided by Microsoft—for encryption at rest and in transit. Other features will be available soon, such as security features integrated with HDInsight and Databricks. Like ADLS Gen1, Gen2 uses POSIX-compliant Access Control Lists and Role Based Access Control, restricting access to only authorized users and authenticating with Azure Active Directory OAuth 2.0 bearer tokens.

"Once authenticated, permissions are applied at the finest granularity to ensure the right level of authorization for protecting an enterprise's big data assets. End-to-end encryption of data and transport layer protections complete the security shield for an enterprise data lake," Baker stated.

Baker emphasized that as part of the Azure Blobs service, ADLS Gen2 will be available in all Azure regions worldwide. Microsoft views this as an important attribute for many companies that want to run their data lakes close to employees, thus benefiting from lower latency, as they work to comply with mounting sovereignty regulations about where data ...