Syntax Highlighter

martes, 12 de enero de 2016

Visual Studio AppInsights module for DNN

DNNLovesAppInsightsAs a website developer or operator, I always need to know if my site goes down and get an alert, verify if the site is performing well or if it is under an attack. There are lot of tools today to give you an insight on what is happening on your web deployment, and one I’m using more and more is Visual Studio Application Insights.

While AppInsights is still a “Preview” service (note that preview means that is not generally available, so no SLA is offered yet), you can start from a Free tier that probably fits the majority of small websites, and optionally start paying depending on the amount of telemetry data you send to the store. This gives to you a powerful insight and tools to operate, diagnose and fix issues immediately.

“With great power comes…great number of alerts!” – David Rodriguez

I’m not going to start selling you all the benefits of using AppInsights or another service such as NewRelic. I’ve been personally using them both for a long time, and while NR has been on the first position on my insights tools list, now I’m getting very excited on what AppInsights is offering today. Here a few interesting highlights for DNN website developers and owners:

  • Monitor the usage and performance of live apps
  • Get immediate alerts on performance or availability issues
  • Get telemetry for existing web apps without redeploying
  • Use for a wide range of app types on devices, servers, or desktops
  • Monitor ASP.NET web apps hosted anywhere: on Azure, other cloud services, or on-premises servers
  • Search traces and exception logs for failure diagnoses (including DNN Event logs and log4net logs!!)
  • Track events, metrics, page views, users, crashes, dependencies, perf counters, and response times

AppInsights module for DNN

The only thing that perhaps is not easy and not documented at all, is how to setup AppInsights on a DNN instance. While I’ve been doing the task manually for a while, I have finally created a simple module available at https://github.com/davidjrh/dnn.appinsights that allows, on this initial version:

  • Easily setup AppInsights on a DNN Platform or Evoq installation as a Host user
  • Automatically send telemetry data to AppInsights:
    • Http requests information
    • Page views
    • Server and browser exceptions
    • Trace information including the log4net logs information
    • DNN event log entries with a new logging provider
    • Performance counters

There is still space for improvement, like adding UI to specify which performance counters you want to add (currently you have to manually edit the ~/ApplicationInsights.config file for this task). Feedback is welcome! Pull requests are welcome!

And that is not all since more features are continuously being added to AppInsights. Can you imagine a service that automatically learns how your site is being used and alert you if an abnormal pattern is detected? You would like to contact AppInsightsML@microsoft.com to try it out!

ProactiveDetection

Getting started

This module is a DNN Platform extension to integrate Visual Studio Application Insights to monitor your DNN installation. To setup the module on your installation, follow these steps:

  1. Provision a new AppInsights service following the guide at https://azure.microsoft.com/en-us/documentation/articles/app-insights-overview/. Ensure you choose "ASP.net web application" on the "Application Type" parameter
    CreateAppInsights
  2. Once provisioned, copy the "Instrumentation Key" available on the resource Essentials properties
    InstrumentationKey
  3. Now from the Releases folder https://github.com/davidjrh/dnn.appinsights/tree/master/Releases, download the latest module package version ending on "...Install.zip" (the Source.zip package contains the source code that is not needed for production websites).
  4. Install the extension package in your DNN instance from the "Host>Extensions" menu like any other module
  5. Once installed, a new menu under "Host (Advanced menu)>Application Insights" will allow you to paste the instrumentation key obtained on step 2. After applying the changes, you will start receiving data on AppInsights after a few minutes.
    ModuleSetup

What changes are done on my site?

Some changes are done during the installation and other when Enabling the Application Insights module. Note that by default, until you enable the module and specify an instrumentation key, any AppInsights module or assembly will be loaded and no telemetry data will be sent.

During the install, the following assemblies will be added to the ~/bin folder:

  • DotNetNuke.Monitoring.AppInsights.dll
  • Microsoft.AI.Agent.Intercept.dll
  • Microsoft.AI.DependencyCollector.dll
  • Microsoft.AI.PerfCounterCollector.dll
  • Microsoft.AI.ServerTelemetryChannel.dll
  • Microsoft.AI.Web.dll
  • Microsoft.AI.WindowsServer.dll
  • Microsoft.ApplicationInsights.dll
  • Microsoft.ApplicationInsights.TraceListener.dll
  • Microsoft.Web.XmlTransform.dll

After enabling the module, the following configuration files are changed, in order to send telemetry, dot4net logs tracing and DNN event logs to AppInsights:

  • /Web.config
  • /ApplicationInsights.config
  • /DotNetNuke.log4net.logs
  • /DesktopModules/AppInsights/js/appinsights.js

If you disable or uninstall the module, all the previous changes are reverted.

The following image illustrates the DNN event log data being sent to AppInsights, where you can search or filter by content, and why not, create some alerts based on any criteria.

EventLogProvider

 

Resources

viernes, 20 de noviembre de 2015

Si te perdiste Connect(); vente a ReConnect(); con TenerifeDev

ReConnectTenerifeDev

Los días 18 y 19 de noviembre hemos podido ver en vivo y en directo el evento Connect 2015, donde desde Nueva York se nos han presentado el presente y futuro de las herramientas y servicios de Microsoft para una nueva era de desarrolladores.

“El papel de los desarrolladores está cambiando dramáticamente en el mundo de hoy, y Microsoft también lo está haciendo. Hace un año comenzamos un viaje para un nuevo Microsoft para desarrolladores, al presentar el futuro de un .NET de código abierto en Linux y Mac, y Visual Studio gratis para apuntar a cualquier dispositivo y sistema operativo.”

Así rezaba la introducción de la KeyNote de Scott Guthrie, en un discurso lleno de anuncios y demostraciones, mostrando los siguientes pasos de la transformación de Microsoft con las herramientas y servicios que ayudan a los desarrolladores a tener éxito en esta nueva era.

ReConnect();

TenerideDev y otras comunidades técnicas a lo ancho y largo del territorio español, hemos querido acercarte la repetición de las jugadas más interesantes en un evento resumen llamado ReConnect(); siguiendo la idea del edición anterior. Para ello, hemos desplazado a enero la próxima charla que teníamos programada para hacerle hueco a este pedazo de evento donde veremos cosas como compilar .NET nativo para Linux desde Visual Studio. What?? Sí, lo que oyes, estos suecos se han vuelto locos.

Evento: ReConnect() 2015
Fecha: 17 de diciembre de 2015
Lugar: Salón de Grados de la Escuela Superior de Ingeniería y Tecnología (anteriormente ETSII)
Horario: 16:30 a 19:30 (con descanso incluido)
Speakers: Santiago Porras (@saintwukong, Windows Platform MVP), Cesar Abreu (@cesabreu, Azure MVP), David Rodriguez (@davidjrh, Azure MVP)

Inscripción: en el MeetUp de TenerifeDev 

¡Corred insensatos!

jueves, 8 de octubre de 2015

Rebuilding SQL Database indexes using Azure Automation

Some days back I had a discussion with an Azure SQL Database engineer about the need of rebuilding indexes on SQL Database. I thought that was a best practice but the engineer told me that the task was accomplished automatically as part of the managed service, in the same way you haven’t had to execute any file related task.

I remembered the thing today and found two interesting articles from Alexandre Brisebois just confirming my initial thoughts, that the indexing management is under your responsibility, and you need to pay attention to how fragmented they are.

There are three good blog posts from Alexandre on the matter I found interesting:

Running the following T-SQL on a SQL Database, you can get the index fragmentation on a specific table in percent:

SELECT name, avg_fragmentation_in_percent

FROM sys.dm_db_index_physical_stats (

       DB_ID(N'MyDatabaseName')

     , OBJECT_ID('MyTableName')

     , NULL

     , NULL

     , NULL) AS a

JOIN sys.indexes AS b

ON a.object_id = b.object_id AND a.index_id = b.index_id

If you want to get all the indexes in a database with more than a 30% of fragmentation, you can run this other one:

SELECT name, avg_fragmentation_in_percent

FROM sys.dm_db_index_physical_stats (

       DB_ID(N'MyDatabaseName')

     , NULL

     , NULL

     , NULL

     , NULL) AS a

JOIN sys.indexes AS b

ON a.object_id = b.object_id AND a.index_id = b.index_id

WHERE avg_fragmentation_in_percent > 30

image

And now don’t be scared when seeing the results. The good news is that you are going to get better performance after rebuilding the indexes without having to scale to another SQL Database tier. Alexandre itself commented something about the possibility of using Azure Automation to do the task.

Reindexing using Azure Automation

I’m in love with Azure Automation and all the small things that can be automated to do our daily job easier, mostly based on running a small script on a schedule. Curiously there is a PowerShell Workflow Runbook on the Automation Gallery that allows to fully automate the SQL Database reindexing. Let’s see how to configure it step by step:

1) Provision an Automation Account if you don’t have any, by going to https://portal.azure.com and select New > Management > Automation Account

image

2) After creating the Automation Account, open the details and now click on Runbooks > Browse Gallery

image

3) Type on the search box the word “indexes” and the runbook “Indexes tables in an Azure database if they have a high fragmentation” appears:

image

4) Note that the author of the runbook is the SC Automation Product Team at Microsoft. Click on Import:

image

5) After importing the runbook, now let’s add the database credentials to the assets. Click on Assets > Credentials and then on “Add a credential…” button.

image

6) Set a Credential name (that will be used later on the runbook), the database user name and password:

image

7) Now click again on Runbooks and then select the “Update-SQLIndexRunbook” from the list, and click on the “Edit…” button. You will be able to see the PowerShell script that will be executed:

image

8) If you want to test the script, just click on the “Test Pane” button, and the test window opens. Introduce the required parameters and click on Start to execute the index rebuild. If any error occurs, the error is logged on the results window. Note that depending on the database and the other parameters, this can take a long time to complete:

image

9) Now go back to the editor, and click on the “Publish” button enable the runbook. If we click on “Start”, a window appears asking for the parameters. But as we want to schedule this task, we will click on the “Schedule” button instead:

image

10) Click on the Schedule link to create a new Schedule for the runbook. I have specified once a week, but that will depend on your workload and how your indexes increase their fragmentation over time. You will need to tweak the schedule based on your needs and by executing the initial queries between executions:

image

11) Now introduce the parameters and run settings:

image

NOTE: you can play with having different schedules with different settings, i.e. having a specific schedule for a specific table.

With that, you have finished. Remember to change the Logging settings as desired:

image

Conclusion

Cool? SUPERCOOL!!

Un saludo y happy coding!

sábado, 12 de septiembre de 2015

Azure Web Apps: Kudu REST API is a box of surprises

VitorinoxAfter working with Azure Websites Web Apps for a long time, I have noticed that every day I use some tips and tricks that surprise people in some way. Normally the tips are nothing spectacullar, but when you use them your daily productivity is enhanced. I’m going to start posting some of these tips in order to have my own “notebook” to revisit when needed, but also to share and make easier small tasks like working with files, website management, getting alerts, etc.

So let’s start with a simple tip:

How to download the website contents of a site hosted on Azure Web Apps

There are many ways to answer this question coming to my mind: FTP, MS Deploy, uploading 7zip command line tool via Kudu, ...but is there something more simple than that just for downloading the content?

Answer is YES.

Azure Web Apps ZIP API

After spending some time working with Azure Web Apps, you have probably noticed that “behind” your website, there is a lot of tools being served by the SCM (Service Control Manager, also known as Kudu).

You can direclty access this service by simply browsing the “https://mywebsite.scm.azurewebsites.net” URL, where “mywebsite” is your site name, and then introducing your Azure credentials (using an Azure AD or Microsoft Account). You can also use Basic Authentication by browsing “https://mywebsite.scm.azurewebsites.net/basicauth” and then introduce the deployment credentials you can get from your website settings at the Azure Management portal.

Kudu offers you an user interface full of tools to manage and diagnostic your web app:

KuduUI

And if you dig into the Kudu documentation, you will notice that some REST APIs come out-of-the-box. One of these REST APIs is the ZIP, that allows downloading folders as zip files or expanding zip files into folders:

ZIPAPI

Download website contents using a simple URL in your browser

Enough! With just introducing this URL in your browser and typing your credentials, you can download your full website contents:

https://mywebsite.scm.azurewebsites.net/api/zip/site/wwwroot

If you want to download a subfolder of your website, you could use something like:

https://mywebsite.scm.azurewebsites.net/api/zip/site/wwwroot/subfolder1

Note that you can download anything inside the “D:\home” folder where the “/api/zip” is relative, so if for example, you want to download all your site log files, including IIS log files, you can use the following URL:

https://mywebsite.scm.azurewebsites.net/api/zip/LogFiles

NOTE: an equivalent one would be to use the “dump” API:

https://mywebsite.scm.azurewebsites.net/api/dump

Adding some Azure PowerShell sauce

Is quite normal to download these log files to your PC and then run your favourite log parsing tool like Log Parser and Log Parser Studio. It’s easy to manually download them from Kudu but it’s not funny when you have to do the same task almost every day over some hundreds of websites.

So why not to use PowerShell to automate the task?

After installing Azure PowerShell, you can run the following script to download the files and folders using the Kudu ZIP REST API. You can tweak it a little by iterating between all your websites, and also iterating all your subscriptions, so you could download the IIS logs of all the websites you own just with some lines of code.

In the following script, I’ve changed the folder to download to a specific one where DNN Platform stores their Log4Net daily logs, that BTW, you can then review on-premises using Log4View.

NOTE: I used PowerShell 5.0 available on Windows 10, with “wget” and “Invoke-RestMethod” support what simplifies the script.

# Input parameters

$subscriptionName = "MySubscriptionName"

$websiteName = "MyWebsitename"

$slotName = "Production"

$folderToDownload = "site/wwwroot/Portals/_default/logs/" # must end with / for folders or you will get a 401

$outputZipFile = "D:\Temp\LogFiles.zip"

 

# Ask for Azure credentials to obtain the publishing credentials

Add-AzureAccount

Select-AzureSubscription $subscriptionName

 

# Build the basic authentication header

$website = Get-AzureWebsite $websiteName -Slot $slotName

$publishingUsername = $website.PublishingUsername

$publishingPassword = $website.PublishingPassword

$base64AuthInfo = [System.Convert]::ToBase64String( `

                    [System.Text.Encoding]::ASCII.GetBytes(( `

                    "{0}:{1}" -f $publishingUsername, $publishingPassword)))

 

# Download the log files using wget or Invoke-RestMethod, available in Windows Powershell 5.0 :)

Invoke-RestMethod -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)} `

                  -Uri "https://$websiteName.scm.azurewebsites.net/api/zip/$folderToDownload" `

                  -OutFile $outputZipFile

 

And don’t forget other Kudu APIs available!

I have some other tips I use in a daily basis I will be posting soon. Don’t forget to take a look to other APIs available, because they are a box of surprises:

  • api/vfs: allows you to execute files and folder operations
  • api/command: allows you to execute something (one of my favourites when combined with the previous one)
  • api/settings: allows you to change the site settings
  • api/dump, api/diagnostics, api/logs: for diagnostics, tracing, etc.
  • api/scm, api/deployments, api/sshkey: for repository and deployment management;
  • api/siteextensions: enable or diable other site extensions like Visual Studio Monaco. The available extensions grow in a monthly basis, don’t forget to revisit from time to time

Un saludo and Happy Coding!

miércoles, 9 de septiembre de 2015

Próximos eventos TenerifeDev

Después de un Agosto sofocante y aún con la resaca de la Tenerife LAN Party, ya estamos de vuelta! Y esta vez vamos a programar los eventos de los próximos cuatro meses en un flash, donde hablaremos de todo un poco: Xamarin, Machine Learning, Windows 10 IoT Core, Raspberry Pi, y hasta una sesión que nos pareció la mar de interesante dedicada íntegramente al personal branding.

¿Cuándo? ¿Dónde?

A continuación mostramos la lista de eventos y sus fechas, que en principio se desarrollarán en el Salón de Grados de la ETSII como viene siendo habitual. Lo más recomendable es que os registréis en http://www.meetup.com/es/tenerifedev/ para estar al tanto de cualquier modificación y del contenido de cada sesión:
  • Jueves 17 de septiembre de 18:00 a 19:30 - Xamarin Forms
  • Jueves 15 de octubre de 18:00 a 19:30 - Machine Learning
  • Jueves 19 de Noviembre de 18:00 a 19:30 - Automatizando la oficina con Raspberry Pi, Windows 10 y Azure
  • Jueves 17 de diciembre de 18:00 a 19:30 - Personal branding: Por qué y cómo convertirnos en marca
¡Corred insensatos!
Related Posts Plugin for WordPress, Blogger...