Clik here to view.
Encoding 101 – Exporting from SQL Server into flat files, to create a Hive...
Today in Microsoft Big Data Support we faced the issue of how to correctly move Unicode data from SQL Server into Hive via flat text files. The main issue faced was encoding special Unicode characters...
View ArticleClik here to view.
How to call a Azure Machine Learning Web Service from NodeJS
Azure machine learning allows data scientists and developers to embed predictive analytics into applications. To learn more about Azure machine learning visit Azure machine learning documentation . A...
View ArticleHDInsight Hive Metastore fails when the database name has dashes or hyphens
Working in Azure HDInsight support today, we see a failure when trying to run a Hive query on a freshly created HDInsight cluster. Its brand new and fails on the first try, so what could be wrong? Our...
View ArticleClik here to view.
HDInsight Name Node can stay in Safe mode after a Scale Down
This week we worked on an HDInsight cluster where the Name Node has gone into Safe mode and didn’t leave that mode on its own. It’s not very common, but I wanted to share why it happened, and how to...
View ArticleClik here to view.
How to Lock a Resource Group to prevent accidental deletion of resources like...
Did you know it is possible to prevent accidental deletion of resources in Azure? This could apply to any number of resource, HDInsight, Stream Analytics jobs, Data Factories, DocumentDB accounts, etc....
View ArticleClik here to view.
How to Find and Kill a running Yarn Application Master in HDInsight with and...
Today we faced a challenge in HDInsight not knowing the SSH user password to terminal into the server, and we needed to kill some running Hive jobs that were too far gone and taking too many resources....
View ArticleClik here to view.
HDFS gets full in Azure HDInsight with many Hive temporary files
Sometimes when Hive is using temporary files, and a VM is restarted in an HDInsight cluster in Microsoft Azure, then those files can become orphaned and consume space. In Azure HDInsight, those temp...
View ArticleClik here to view.
Capture Microsoft Azure Stream Analytics logs
Microsoft Azure Stream Analytics makes building real time solution very easy. Developers can build Stream Analytics job with few clicks. While running Stream Analytics jobs, you may encounter an error...
View ArticleClik here to view.
Rerunning many slices and activities in Azure Data Factory
Today someone asked me how to run all the data slices in their data factory on-demand in an ad-hoc fashion, to say run the whole pipeline again from scratch. For example, if you have a one-time copy...
View ArticleClik here to view.
Encoding the Hive query file in Azure HDInsight
Today at Microsoft we were using Azure Data Factory to run Hive Activities in Azure HDInsight on a schedule. Things were working fine for a while, but then we got an error that was hard to understand....
View Article