Skip to main content

Configuring secondary database in Azure SQL DB - Bug found

Hi All

Last week we had an issue with a secondary DB in geo replication and fail over group.

To make the long story short we had to delete the secondary and recreate a secondary from scratch.

And now let me tell you the story, we build a DB in P6 tier - very high, expensive and highly available.

Then we add a geo replication copy via the platform, like it shows here.



This is take from MSFT documentation:

https://docs.microsoft.com/en-us/azure/azure-sql/database/active-geo-replication-overview

It is written: 

"Both primary and secondary databases are required to have the same service tier. It is also strongly recommended that the secondary database is created with the same backup storage redundancy and compute size (DTUs or vCores) as the primary. If the primary database is experiencing a heavy write workload, a secondary with lower compute size may not be able to keep up with it. That will cause redo lag on the secondary, and potential unavailability of the secondary. To mitigate these risks, active geo-replication will throttle the primary's transaction log rate if necessary to allow its secondaries to catch up.

Another consequence of an imbalanced secondary configuration is that after failover, application performance may suffer due to insufficient compute capacity of the new primary. In that case, it will be necessary to scale up database service objective to the necessary level, which may take significant time and compute resources, and will require a high availability failover at the end of the scale up process.

If you decide to create the secondary with lower compute size, the log IO percentage chart in Azure portal provides a good way to estimate the minimal compute size of the secondary that is required to sustain the replication load. For example, if your primary database is P6 (1000 DTU) and its log write percent is 50%, the secondary needs to be at least P4 (500 DTU). To retrieve historical log IO data, use the sys.resource_stats view. To retrieve recent log write data with higher granularity that better reflects short-term spikes in log rate, use sys.dm_db_resource_stats view." 

So we have decide to create the secondary with lower compute size. its nor recommend - but we can live with it.

We have created a P4 secondary and all went very well.

We used in those DB's an in-memory table, lately its been growing. one day we say in the monitoring that the secondary is stuck. So as a cloud lovers we just click to scale it up. Nothing happened. We clocked again.... nothing happened.

So we opened a ticket, they also tried to understand what wend wrong.

They found the root cause but did not find a way to solve it.

The in-Memory Table in the primary went higher then the size permitted in the secondary. Scale up stuck all time

Nothing helped, we need the secondary urgently, so we decided to cut it from the geo-replication and delete it and build a new secondary.


Those are the limitation :

https://docs.microsoft.com/en-us/azure/azure-sql/database/resource-limits-dtu-single-databases




I know we need to work by the recommendations but i am sure this scenario just now added to the automation QA :-).



Labels:

Comments

New comments are not allowed.

Popular posts from this blog

Extended Events in SQL Azure

Hi Everybody   Today an English post about 'Extended Events in SQL Azure', some of you shorten the name to 'EE' and some to 'XEvent'. I Love EE so this is how I will call it in this post.   This feature was introduce in SQL Server 2008 and its should help collecting DATA about what is running in the Server.   More Details about this SQL Server feature can be found in this Link: https://msdn.microsoft.com/library/bb630282.aspx?f=255&MSPPError=-2147217396   There are a few differences between EE in SQL Azure and regular SQL Server: In SQL Server versions the EE are on the Server level and therefore you create sessions on Server. In SQL Azure the server is a virtual entity - so the EE is in DB level and you create the session on DataBase. In SQL Server versions the EE can write to files on the server. SQL Azure does not have drives for files (SQL Azure is PAAS.....:-)). There is an option to write to blob storage, for this we need t...
Post a Comment
Read more

How to restore deleted Azure Synapse dedicated SQL pool

  Existing dedicated pool can be easily restored from Azure portal or PowerShell command, but for now deleted pool could be restored from PowerShell only! Example: # Connect to Azure with system-assigned managed identity $AzureContext = (Connect-AzAccount -Identity).context # set and store context $AzureContext = Set-AzContext -SubscriptionName $AzureContext.Subscription -DefaultProfile $AzureContext # $AzureContext = Set-AzContext -SubscriptionName $SubscriptionName -DefaultProfile $AzureContext $SubscriptionName="Databases" $ResourceGroupName="stg-rg-we" $ServerName="stg-synapse-we"   $DatabaseName="sql_we_2023_11_07_13_42" $NewDatabaseName="sql_dp_we_deleted" ######################################## $token = (Get-AzAccessToken -ResourceUrl https://database.windows.net).Token $SubscriptionId = "ce088f9e-1111111a3914b" $DedicatedPoolEndPoint = "stg-synapse-we.sql.azuresynapse.net" $DedicatedPoolName = $DatabaseNam...
Post a Comment
Read more

The journey to the Lakehouse

A long time has passed since the last post, we have gone through a long and tedious journey to adapt what Azure offers us, to our needs. Our needs were simple, the Current Datawarehouse (SQL Server on VM inazure) served the BI. ML teams worked on GCP, we want to let both teams to work on Azure in a platform that will have the ability to scale and will not fail every 2 days. We checked: Snowflake on azure Synapse analytics GCP We decided to go for the full Azure product for the reasons: Migration time support costs Synapse as a platform contains many components, and the challenge was to find what fits  us as an organization and as a group. The knowledge of the people and their abilities influenced the plans. Here's what we planned and what we did: We start to put everything in the Data Lake in parquet or delta format, build on top of Azure ADLS gen 2. We had to move some data to T-SQL compatible platform, so this involves setting up a dedicated Synapse pool , which is a fully man...
Post a Comment
Read more