SQL Azure for DBAs and Data Engineers

  Existing dedicated pool can be easily restored from Azure portal or PowerShell command, but for now deleted pool could be restored from PowerShell only! Example: # Connect to Azure with system-assigned managed identity $AzureContext = (Connect-AzAccount -Identity).context # set and store context $AzureContext = Set-AzContext -SubscriptionName $AzureContext.Subscription -DefaultProfile $AzureContext # $AzureContext = Set-AzContext -SubscriptionName $SubscriptionName -DefaultProfile $AzureContext $SubscriptionName="Databases" $ResourceGroupName="stg-rg-we" $ServerName="stg-synapse-we"   $DatabaseName="sql_we_2023_11_07_13_42" $NewDatabaseName="sql_dp_we_deleted" ######################################## $token = (Get-AzAccessToken -ResourceUrl https://database.windows.net).Token $SubscriptionId = "ce088f9e-1111111a3914b" $DedicatedPoolEndPoint = "stg-synapse-we.sql.azuresynapse.net" $DedicatedPoolName = $DatabaseNam...

A long time has passed since the last post, we have gone through a long and tedious journey to adapt what Azure offers us, to our needs. Our needs were simple, the Current Datawarehouse (SQL Server on VM inazure) served the BI. ML teams worked on GCP, we want to let both teams to work on Azure in a platform that will have the ability to scale and will not fail every 2 days. We checked: Snowflake on azure Synapse analytics GCP We decided to go for the full Azure product for the reasons: Migration time support costs Synapse as a platform contains many components, and the challenge was to find what fits  us as an organization and as a group. The knowledge of the people and their abilities influenced the plans. Here's what we planned and what we did: We start to put everything in the Data Lake in parquet or delta format, build on top of Azure ADLS gen 2. We had to move some data to T-SQL compatible platform, so this involves setting up a dedicated Synapse pool , which is a fully man...

I recently moved a role From being a DBA Manager, Who is responsible for the operational databases. I moved to manage the data engineering group. So what exactly is the difference between the two functions? DBA - Production Databases: SQL\ NoSQL- 24*7, powerful server on premise or on the cloud, managed or semi managed, security tasks, high performance is a target, multiregional, HA as top priority. Developers are using Microservices - so we have many applications many services and many many Databases. Many kinds of DB's like Cloud IAAS and PAAS. Secure and audit the data is must. The Clusters must have Uptime as long as we can achive. Data Modeling - is so important too. Challenges and Problems in the data bases systems Lots of DB’s Lots of creators / no standards Lots of Consumers (Query, tools, SLA) Raw data Lots of data resources Data silos In Data Engineering we have other challenges for example we have Data lake and Data Warehouses : Batch process. Stream Process. many data ...

Hi All Last week we had an issue with a secondary DB in geo replication and fail over group. To make the long story short we had to delete the secondary and recreate a secondary from scratch . And now let me tell you the story, we build a DB in P6 tier - very high, expensive and highly available. Then we add a geo replication copy via the platform, like it shows here. This is take from MSFT documentation: https://docs.microsoft.com/en-us/azure/azure-sql/database/active-geo-replication-overview It is written:  " Both primary and secondary databases are required to have the same service tier. It is also strongly recommended that the secondary database is created with the same backup storage redundancy and compute size (DTUs or vCores) as the primary. If the primary database is experiencing a heavy write workload, a secondary with lower compute size may not be able to keep up with it. That will cause redo lag on the secondary, and potential unavailability of the secondary. To mit...

שלום לכולם כאשר מקנפגים Azure SQL DB מנוהל והוא תחת Premium or Business Critical Toers מיקרוסופט שואלים אותך: Would you like to make this database zone redundant? מה זו השאלה הזו? מה ההשפעה שלה ומה העלות שלה? אז עשינו ובדקנו מה ההשפעה. הלינק הבסיסי להסבר הקונפיגורציה הזו הוא זה: https://azure.microsoft.com/is-is/blog/azure-sql-database-now-offers-zone-redundant-premium-databases-and-elastic-pools/ מה זה אומר? זה מסביר על המושג הקריטי Availability Zones ועל השימוש שלו ב SLA של הדיבי שלנו. ובכן בכל דאטה סנטר בענן יש בעצם 3 דאטה סנטרים, הכל מנותק אחד מהשני. https://docs.microsoft.com/en-us/azure/availability-zones/az-overview לכל Azure SQL DB יש 3 עותקים ניסתרים, מיקרוסופט נותנת אופציה לשמור את אחד העותקים הניסתרים ב Availability Zone אחר. הדבר מעלה את ה SLA כך שאם יש תקלה ב Availability Zone אחד זה לא משפיע על ה DB. לכן אני ממליץ בחום לכולם לאפשר את הקונפיגורציה הזו.

Hi All   New feature was released lately - Fail Over in the Geo Replication in SQL Azure. The Announcement is here:   https://azure.microsoft.com/en-us/blog/spotlight-on-new-capabilities-of-azure-sql-database-geo-replication/   As You can see they added Failover feature. That means that until today you have Primary and Secondary DB's, DATA moved on one direction and you could disconnect the connection when ever you wish. but then you have to build everything from the beginning, (you can look in my old post about geo-replications types in sql azure). Now DATA moved backward, when you click Fail over - you still have 2 DB's but the Secondary become Primary and vise versa. I this this is a great feature. you can also read in Hebrew about it in Geri's Reshef Blog   http://blogs.microsoft.co.il/gerireshef/2015/11/23/%D7%90%D7%95%D7%A4%D7%A6%D7%99%D7%99%D7%AA-geo-replication-%D7%91-azure/   This is the new link   enjoy   ...

Hi Everybody   Today an English post about 'Extended Events in SQL Azure', some of you shorten the name to 'EE' and some to 'XEvent'. I Love EE so this is how I will call it in this post.   This feature was introduce in SQL Server 2008 and its should help collecting DATA about what is running in the Server.   More Details about this SQL Server feature can be found in this Link: https://msdn.microsoft.com/library/bb630282.aspx?f=255&MSPPError=-2147217396   There are a few differences between EE in SQL Azure and regular SQL Server: In SQL Server versions the EE are on the Server level and therefore you create sessions on Server. In SQL Azure the server is a virtual entity - so the EE is in DB level and you create the session on DataBase. In SQL Server versions the EE can write to files on the server. SQL Azure does not have drives for files (SQL Azure is PAAS.....:-)). There is an option to write to blob storage, for this we need t...

שלום לכולם והיום על versions. קיימים שלושה מספרים המייצגים את version של ה sql שלך: Select @@version -->  Microsoft SQL Azure (RTM) - 12.0.2000.8   Sep  2 2015 20:51:51   Copyright (c) Microsoft Corporation  select  DATABASEPROPERTYEX(db_name(), 'Version')  --> number like 826 or any other The Performance tier שלושה מספרים אלו מייצגים את מספר גרסת ה sql ברמת בסיס נתונים כאשר ייתכן שתהיה אותה ורסיה אולם מספר שונה - ואפילו באותו שרת ייתכנו שינויים בין בסיס נתונים. למה זה חשוב? לא באמת זה חשוב ולא באמת זה מועיל - אבל יש פה כמה היבטים - ראשית אפשר ללמוד הרבה מהגלגולים ופריסת הגרסאות הרבה ויחד עם זאת אפס אחוז  downtime. שנית כאשר ישנה תקלה יותר נוח למסור ישר את 3 המספרים אלו ואז הטיפול יהיה ממש מהיר - הם יוכלו לזהות במהירות מצי נפרסה הגרסה וכך לזהות בעיות שיכלו להתרחש. ועכשיו ארחיב לגבי הנקודה השלישית. אם תיראו בפוסט הקודם הזכרתי את 3 הגרסאות שקיימות היום ב SQL Azu...