PowerShell command to get the metadata

Here is the command to get the metadata of all of your blobs in your storage account.

Set your execution policy to Process if you get any policy errors.

Set-ExecutionPolicy -ExecutionPolicy ByPass -Scope Process

If you get an authorization error as follows, make sure to add a role assignment of Storage Blob Data Contributor. You can do this via Access Control (IAM) of your storage account. Read more about this role here.

PowerShell AuthorizationPermissionMismatch Error

Get-AzStorageContainer : This request is not authorized to perform this operation using this permission.
RequestId:405a7e1a-f01e-004f-3acd-6a1168000000
Time:2022-05-18T15:38:30.5034936Z
Status: 403 (This request is not authorized to perform this operation using this permission.)
ErrorCode: AuthorizationPermissionMismatch
Content:
<?xml version=”1.0″ encoding=”utf-8″?><Error><Code>AuthorizationPermissionMismatch</Code><Message>This request is not authorized to perform this operation using
this permission.

Once you add the role, you can try running your PowerShell command after a few minutes. Please make sure to provide the values to the variables $containerName, $subscriptionid, $storageAccName. Happy coding with PowerShell!

About the Author

I am yet another developer who is passionate about writing and video creation. I have written close to 500 blogs on my blog. If you like this content, consider following me here,

• GitHub
• medium
• Twitter

Your turn. What do you think?

Thanks a lot for reading. Did I miss anything that you may think is needed in this article? Could you find this post useful? Kindly do not forget to share your feedback.

Kindest Regards

Sibeesh Venu

Install-Module -Name AzureAD

You will be prompted to accept the installation, give Yes or A.

Untrusted repository
You are installing the modules from an untrusted repository. If you trust this repository, change its InstallationPolicy value by running the Set-PSRepository cmdlet. Are you sure you want to install the modules from 'PSGallery'?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): A

You can also check whether this module is installed or not by using the command Get-Module AzureADPreview

Once you are done, it is time to connect to your Azure AD by using the command Connect-AzureAD. That will give you an output as in the preceding image.

Now, use the command below to change the properties of your user.

Set-AzureADUser -ObjectId user1@domain.com -OtherMails @("user2@domain.com") -Mobile "+353 000000000" -TelephoneNumber "+353 000000000"

You can also verify the changes by running the commands below.

Get-AzureADUser -ObjectID user@domain.com | select otherMails
Get-AzureADUser -ObjectID user@domain.com | select Mobile
Get-AzureADUser -ObjectID user@domain.com | select TelephoneNumber

After you have changed your mobile number, verify the same when it is asked, and you should be able to Enable the Two-Step Verification on the Security Information page.

References

https://docs.microsoft.com/en-us/azure/active-directory/authentication/howto-sspr-authenticationdata#use-powershell-version-2

Introduction

When you have an on premises Virtual Machine and if you need to move the same to a cloud, you should create a Virtual Hard Disk. The virtual hard disks are of two types, one is VHD and the other is VHDX. Unfortunately the Microsoft Azure cloud supports only the generation 1 VMs which are VHD, These VHDs are fixed size and the maximum size allowed is 1023 GB. Here in this article, we are going to see how we can spin up a Virtual Machine in Azure from the VHD file we have. I will explain why I came to this situation in the Background section. I hope you will find this article useful.

Background

I was working with a product called UCS (Univention Corporate Server), you can consider it as a private play store where you can install and use your custom applications, you are also allowed to use the existing ones as well. Now I wanted to create an Azure VM with the UCS in it, but all I had in my hand was a VHDX file, which is not supported in Azure.

Steps

In this article, we will be doing the following tasks.

1. Converting the VHDX file to VHD
2. Upload it to the Azure Blob storage
3. Create a Managed Image from the VHD
4. Create the Virtual Machine from the Managed Image

Spin up the Azure VM from VHDX

As we have discussed, the first step is to convert the VHDX file to VHD.

Convert VHDX to VHD

We are going to use PowerShell for the conversion as it has some modules already to support this conversion.

Convert-VHD –Path c:\CreateVM\ucs.vhdx –DestinationPath c:\CreateVM\ucs.vhd -VHDType Fixed

Please be noted that the Argument -VHDType is very important, otherwise you may be getting an error as follows, when you try to create the Managed image using the command New-AzImage.

New-AzImage : Only blobs formatted as VHDs can be imported.
ErrorCode: InvalidParameter
ErrorMessage: Only blobs formatted as VHDs can be imported.
ErrorTarget: disks
StatusCode: 400
ReasonPhrase: Bad Request
OperationID : 16baa116-be93-4054-8ba9-265eda636f0c
At line:1 char:10
+ $image = New-AzImage -ImageName $imageName -ResourceGroupName $rgName ...
+          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [New-AzImage], ComputeCloudException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.Automation.NewAzureRmImage

Once you are done a new VHD image will be created in the destination path you have given. Please be noted that the size of this file may be huge, and when you upload it to the Blob storage, it may take a longer depends on the network speed you have.

Upload the VHD file to Azure Blob Storage

As a prerequisite, you should create a storage account before you start this process. Once you have created the storage account, you are good to go and start the upload. Go to your Storage account, and click on the Blobs under blob services.

Now you can click on the +Container menu item and create a new container which is nothing but the container for your files. Now we can go inside the container and click on the Upload button. In the upcoming pop up you can select the VHD file using the file picker option and upload the file. As I said earlier, this process is totally depends on your network speed.

Sometimes, due to the browser inactivity, the uploading of the heavy files to the blobs using Azure portal can be a headache as it throws error. One of the suggestion I can give you here is to use Azure Storage Explorer tool, which is quite easy to use and effective. Once you download the tool, login with your Azure credentials and got to the storage accounts and click on Blob Containers. In the new window click on Upload and then Upload files.

Please make sure that you are selecting the Blob type as Page Blob and then click upload. The uploading is really faster if you are using the tool instead of the portal.

Create a Managed Image from the VHD file

As we have our VHD file in our Blobs, now it is time to create an Image from the file. We are going to write some PowerShell commands to do the work for us. To set the configuration we are going to use the command New-AzImageConfig. If you are getting the error “The term New-AzImageConfig ‘ is not recognized as the name of a cmdlet”, you should install and import the module Az.Compute.

Install-Module Az.Compute
import-module Az.Compute

It is possible to get some additional error when you try to install the Az.Compute module. In that case, you can try the preceding command.

 Install-Module Az.Compute -AllowClobber

Once that is done, we can set some variables in the PowerShell as follows.

$vmName = "ucs"
$rgName = "ucs"
$location = "West Europe"
$imageName = "ucs"
$osVhdUri = "https://ucs.blob.core.windows.net/files/ucs.vhd"

Now we can start the image creation as preceding.

$imageConfig = New-AzImageConfig -Location $location
$imageConfig = Set-AzImageOsDisk -Image $imageConfig -OsType Linux -OsState Generalized -BlobUri $osVhdUri
$image = New-AzImage -ImageName $imageName -ResourceGroupName $rgName -Image $imageConfig

As the UCS is a Linux based system, I had given the -OsType as Linux, you may have to change it as per your needs. If you are trying to run the command New-AzImage with the VHDX blob, you will get an error as below.

New-AzImage : Blob name in URL https://ucs.blob.core.windows.net/files/ucs.vhdx must end with ".vhd" extension.
ErrorCode: InvalidParameter
ErrorMessage: Blob name in URL https://ucs.blob.core.windows.net/files/ucs.vhdx must end with ".vhd" extension.
ErrorTarget: blobUri
StatusCode: 400
ReasonPhrase: Bad Request
OperationID : 668b8e66-85b3-4c7b-a182-4f155e16b66d
At line:1 char:10
+ $image = New-AzImage -ImageName $imageName -ResourceGroupName $rgName ...
+          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [New-AzImage], ComputeCloudException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.Automation.NewAzureRmImage

But we don’t need to worry about it as we already converted our VHDX file to VHD format. Once the New-AzImage command is successful, you can see the image in the Azure portal if you go to the Images section. You can search for the keyword “Images” in the portal.

Create a VM from the Managed Image

As we have the image now, we can create our VM easily from the Azure portal. Go to the Images and select your image and then click on the button +Create VM. In the next page, you can give the details of your VM. You can also create the VM by using the PowerShell command as preceding.

New-AzVm `
    -ResourceGroupName "myResourceGroup" `
    -Name "myVMfromImage" `
    -ImageName "myImage" `
    -Location "East US" `
    -VirtualNetworkName "myImageVnet" `
    -SubnetName "myImageSubnet" `
    -SecurityGroupName "myImageNSG" `
    -PublicIpAddressName "myImagePIP" `
    -OpenPorts 3389

Once you had filled all the details, you can either click on the Review+Create or you can go through each step. The creation of VM can take a few minutes. Once the task is done, you can go to your Virtual Machine and spin up the same. I would also recommend you to assign a static IP and DNS name to your VM.

Conclusion

In this article, we have learned,

• What are VHDX and VHD files
• How to convert the VHDX file to VHD
• How to upload VHD files to Azure Blob storage
• How to create Managed Image from the VHD file in Azure
• How to create Virtual Machine using the Managed Image

Your turn. What do you think?

Thanks a lot for reading. Did I miss anything that you may think which is needed in this article? Could you find this post as useful? Kindly do not forget to share me your feedback.

Kindest Regards
Sibeesh Venu

Introduction

In my previous article, we have seen how to Set Up Your Raspberry PI with Microsoft Windows 10 IoT Core. Yes, we were successful in flashing the Microsoft Windows 10 IoT Core OS to our Raspberry PI, but let’s try something very handy this time. The main topic we are going to discuss here in this article is SD Card Provisioning. I understand that you have one Raspberry PI with a working OS in it, whether it is a Microsoft IoT  OS or Raspbian. And you might also have an application which runs in your Raspberry Pi. That’s good. But what if you have 100 Raspberry PI and you need to set up your Raspberry PIs for a production environment? Did you get the difficulties in your mind? In this article, we are going to start a series of articles on SD Card Provisioning. For now, let’s just start doing some amazing things. Let’s just start then.

Background

Yeah, I am glad that you have a Raspberry PI with Windows 10 IoT core OS. While setting up your OS, you might have downloaded the IoT Dashboard and flashed the IoT OS to your SD card manually. As I mentioned in the Introduction, this is not a normal scenario when you want to connect multiple (let’s say 100) Raspberry PIs. Flashing the OS by using the Dashboard application might not be a good idea in this case. What should we do then? That’s where PowerShell come into the play. Yes, we are going to try out some PowerShell commands to do all those mentioned jobs for us.

Prerequisites

To do the wonderful things, we always need some prerequisites.  You can always see all the prerequisites from here. But, in this part, we are going to be needed only the preceding tools.

1. Windows Assessment and Deployment Kit (Windows ADK)
2. Windows Driver Kit (WDK) 10
3. IoT Core ADK Add-Ons
4. Windows 10 IoT Core Packages
5. The Raspberry Pi BSP

Creating the Base Windows IoT FFU Image

I assume that you had already gone through the Prerequisites and followed the instructions. If not, please check again.

Create a Workspace

To get started here, please go to the cloned location of the repository iot-adk-addonkit, where you will find a Windows Command Script IoTCorePShell.cmd. Double click on the same file will open a PowerShell with administrative privilege. Now, this is where the game starts. Once it opens it will do some initial process for us.

Running IoTCorePShell.cmd

Now let’s create a workspace now.

New-IoTWorkspace C:\Workspacefolder oemname arm
(or) new-ws C:\workspacefolder oemname arm

new-ws command

Now we have created a workspace successfully. Please be noted that the BSPPKG_DIR is “C:\Program Files (x86)\Windows Kits\10\MSPackages\retail\arm\fre”. I strongly recommend you to check that folder whether you have contents in it or not if there is no content you might haven’t installed the packages mentioned in the prerequisites.

Build BSP

Now it is time to think about the BSP aka Board Support Package. As we already know that we are using Raspberry PI, we can directly download the RPi_BSP.zip file from the GitHub release page. Then you can perform the below commands.

Import-IoTBSP RPi2 C:\Downloads\RPi_BSP.zip
(or) importbsp RPi2 C:\Downloads\RPi_BSP.zip
buildpkg RPi2

If everything is right, you should see an output as below.

Import Raspberry BSP

Build Packages

Once you have created the workspace and extracted the BSP, it is time to build all of our packages.

New-IoTCabPackage All
(or) buildpkg all

If you miss this step before you go to the next step, you will get an error as follows when you build your image.

info: Trying to load file 'C:\OEEWorkspace\Build\arm\InputFMs\OEMFMFileList.xml' as a FM file list ... fatal error : Error: Missing package: C:\OEEWorkspace\Build\arm\pkgs\test.OEM.Sample.cab
info: Trying to load file 'C:\OEEWorkspace\Build\arm\InputFMs\RPi2FMFileList.xml' as a FM file list ... fatal error : Error: Missing package: 

You will also end up in the similar GitHub issue. Trust me, finding the root cause of this issue took me some hours and at the end, I answered my findings here in StackOverflow.

Building all the packages

Once the build is successful, you should be able to see the cab files in your workspace location, in my case, it is C:\OEE\Build\arm\pkgs.

Create a Product

Now let’s create a new Product, consider this as the device for which we are building an image for. We will be using the BSP we have extracted for this step.

Add-IoTProduct ProductA RPi2
(or) newproduct MyProductA RPi2

You will be asked the Manufacturer Name (OEM Name), Family, SKU, BaseboardManufacturer, and BaseboardProduct. You can get the BaseProductName from your Workspace location, in my case C:\OEE\Source-arm\BSP. You can see the sample command here in the preceding image.

Creating a new Product

The command shown in the image will generate a new Product in the Workspace location C:\OEE\Source-arm\Products\OEEIoTCore.

Add-IoTProduct Output

Build the Image

We are almost done, let’s eject all the removable devices from the system and then run the below command.

New-IoTFFUImage <product name> Test
(or)buildimage <product name> Test

This command will give you the FFU image with your base image.

Building Image and Generate FFU File

You can also see the logs in your workspace location in the build folder. The generated FFU image(Flash.ffu) will be available at C:\OEE\Build\arm\OEEIoTCore\Test folder.

Writing/Flashing the FFU Image to SD Card

Wow!, now we have a fully functional FFU image and the only thing pending is to write some bat commands to flash it to SD card, you can use the DISM. Let’s create a text file called format.txt and write some commands as preceding.

select disk 1
clean
create partition primary
select partition 1
active
format fs=fat32 quick
assign
exit

As you can see that we are selecting the disk 1, considering the fact that only one removable disk is been connected to your machine.  Then we format the disk. This format.txt file will be used in our bat file write.bat.

echo off
REM Start formatting
diskpart /s format.txt >log.txt

REM Start flashing
dism.exe /Apply-Image /ImageFile:Flash.ffu /ApplyDrive:\\.\PhysicalDrive1 /SkipPlatformCheck /Quiet

ECHO Finish

The logs will be added to the file called log.txt. The command ImageFile:Flash.ffu is the place where we assign our FFU image. Now let’s just go to the folder where these three files are been placed and open a command prompt in administrative power.

• write.bat
• format.txt
• flash.ffu

Flash IoT Core by Commands

Connect SD Card to Raspberry PI and then Raspberry PI to the Network

Now that our OS is flashed and it is time to connect the SD card to the Raspberry PI and connect a monitor, mouse, keyboard, ethernet to the Raspberry PI. You will see the default IoT core application running. You can also check in the device portal using the IoT Dashboard application, if you are not sure how, please read my previous post.

Conclusion

Wow!. Now we have learned,

• how we can generate the IoT core FFU image using PowerShell
• how to write the same to an SD card.

Is that all? No, in the next article we will learn

• how we can add our app package to this FFU image which can be used in many Raspberry PI
• how to generate the FFU image on demand with some chained PowerShell commands.
• how to use other mechanisms to write the FFU image to the disk

Your turn. What do you think?

Thanks a lot for reading. Did I miss anything that you may think which is needed in this article? Could you find this post as useful? Kindly do not forget to share me your feedback.

Kindest Regards
Sibeesh Venu

Introduction

I understand that you need to move all of your contents to your destination site from your source site in SharePoint online. You might have done some search in Google and saw this post. Here in this article, I will be walking you through the steps you need to perform the SharePoint PnP provisioning. With the latest SharePoint modern experience, Microsoft had decided to turn off some features, and one among them was Custom Scripting. Unfortunately, to do the SharePoint provisioning, we will have to enable the Custom Scripting. Enough talking, let’s jump into the coding. I hope you will find this post useful.

Before you start Provisioning your site

Before you jump into the process of provisioning there is some configuration which you must perform.

• Enable the Custom Scripting
• Set the User as a Term Store Admin

Enable the Custom Scripting

With the latest release, Microsoft had deactivated this feature from the modern SharePoint experience thus we would have to enable it manually. According to Microsoft, below are the features which will not be available until you manually enable the custom scripting.

1. Save Site as Template
2. Save document library as template
3. Solution Gallery
4. Theme Gallery
5. Help Settings
6. HTML Field Security
7. Sandbox solutions
8. SharePoint Designer
9. Uploading files that potentially include script

In our case, we wanted to do SharePoint PnP provisioning, and it is interesting that it is not mentioned anywhere that we need to enable custom scripting to do SharePoint PnP provisioning. You can always enable the Custom Scripting in two ways, one is using the portal, and another is to use PowerShell commands. I would prefer using the PowerShell commands as it will enable the custom scripting right away. If you enable the custom scripting via the portal, it will take 24 hours to get it activated.

To allow custom script on other SharePoint site collections, you will have to download the download the latest SharePoint Online Management Shell. Once you are done you can perform the below commands.

Connect-SPOService -Url https://yoursharepoint-admin.sharepoint.com

The above command will connect your SharePoint admin site, please be noted that you should have a tenant admin access to do this. If you don’t have the access, you should consider getting the access or you can request the tenant admin to do the same.

Once you are connected, you can perform the preceding command.

Set-SPOsite https://yoursharepinttenant.sharepoint.com/sites/yoursharepointsite -DenyAddAndCustomizePages 0

The above command will enable the custom scripting for the site you have mentioned. To check this, you can always go to any of your site collection/list settings, and if you see the option save the list as a template, it is done and you can proceed.

Save list as the template

Set the User as a Term Store Admin

Once we have enabled the custom scripting it is time to add you as the Term Store Admin, again you can always reach out to your Tenant admin to add you as a Term Store Admin. You can ask your Tenant admin to perform the preceding steps.

1. Sign in to Office 365 as a global admin or SharePoint admin.

2. Click on the app launch icon( 9 dots) and then click on Admin to open the Microsoft 365 admin center.
3. Select Admin centers and then click on SharePoint.
4. Now, you should see an option as Term Store and click on it.
5. He/She can add your mail id in the text box near to the heading Term Store Administrators

Once it is done you can always go to your Site Settings > Term Store Management > Term Store Administrators.

Start SharePoint PnP Provisioning

Wow!. We have done all the configuration and now we can start doing the provisioning.

Connect your source site

The first thing to do is, to connect the source SharePoint site, you can do that by using the preceding command.

Connect-PnPOnline https://youradmin.sharepoint.com/sites/yoursite/

Generate a provisioning template

Now we can generate a provisioning template from the web. You can do that by doing the command below.

Get-PnPProvisioningTemplate -Out c:/Provisioning/template.xml

Please be noted that this will create a new folder “Provisioning” in the C drive and then place the template.xml in it. You can always see the other options of Get-PnPProvisioningTemplate here. This command will pull almost everything from the source site, and you can always choose what exactly you need to take from the source site. This will not retrieve the data, you will have to use another command for that.

Add data to the provisioning template

As we have seen that the Get-PnPProvisioningTemplate will not retrieve the data with the list, we will have to use Add-PnPDataRowsToProvisioningTemplate command to do that magic.

Add-PnPDataRowsToProvisioningTemplate -Path c:/Provisioning/template.xml -List 'The list name' -Query '<view></view>'

Please see here to see the other options you can use with the Add-PnPDataRowsToProvisioningTemplate command. Please be noted that the size of the template.xml file will be larger as per the data you have in the source list.

Apply provision to your destination site

Now we have an XML file which has everything we need to move. Let’s just do it then. Connect your destination site by following the command below.

Connect-PnPOnline https://youradmin.sharepoint.com/sites/yourdetinationsite

Please remember to use the same id and password of your Term Store Admin account.

Once you are connected to the destination site, you can perform the Apply-PnPProvisioningTemplate as follows.

Apply-PnPProvisioningTemplate -Path C:/Provisioning/template.xml

If you are getting an error as ” Apply-PnPProvisioningTemplate : The parent content type specified by content type identifier does not exist”, you will have to enable the SharePoint Server Publishing feature in the Site Settings > Site Features. Remember to enable the SharePoint Server Publishing Infrastructure under Site Settings > Site Collection Features before you enable the SharePoint Server Publishing feature in the Site Settings. It is a dependency. You can always see my StackExchange answer here.

Apply-PnPProvisioningTemplate : The parent content type specified by content type identifier does not exist.
At line:1 char:1
+ Apply-PnPProvisioningTemplate -Path C:\Provisioning\template.xml
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (:) [Apply-PnPProvisioningTemplate], ServerException
    + FullyQualifiedErrorId : EXCEPTION,SharePointPnP.PowerShell.Commands.Provisioning.ApplyProvisioningTemplate

Please be noted that you will also get an error as “WARNING: ListInstance Style Library is a Style Library of a site with NoScript enabled and will be skipped” if the custom scripting is not enabled.

You will also get an error as “Access denied. You do not have permission to perform this action or access this resource” if you don’t have Term Store Admin permission.

Apply-PnPProvisioningTemplate : Access denied. You do not have permission to perform this action or access this resource.
At line:1 char:1
+ Apply-PnPProvisioningTemplate -Path C:/Provisioning/template.xml
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : WriteError: (:) [Apply-PnPProvisioningTemplate], ServerUnauthorizedAccessException
    + FullyQualifiedErrorId : EXCEPTION,SharePointPnP.PowerShell.Commands.Provisioning.ApplyProvisioningTemplate

Once you run the Apply-PnPProvisioningTemplate command, and if everything is fine, you will get everything in your destination site. It will take some time to generate everything for you and add the data to it, depends on the amount of the data you have the process can take from 5 minutes to hours.

Conclusion

Congratulation, we have successfully moved all of our data and almost everything from one SharePoint site to other. In this article, we have learned how to,

1. Enable Custom Scripting in SharePoint
2. Add a Term Site Admin
3. Connect to a SharePoint site
4. Create a PnP provisioning template from the web
5. Add data to your Provisioning template form the lists
6. Apply the provisioning to the destination SharePoint site

Your turn. What do you think?

Thanks a lot for reading. I will come back with another post on the same topic very soon. Did I miss anything that you may think which is needed? Could you find this post as useful? Kindly do not forget to share me your feedback.

Kindest Regards
Sibeesh Venu