Getting Linux Running on WSL2 (Windows)

I haven’t touched Linux in many years and after seeing Microsoft’s Build this year, I thought it would be a fun project to get into.  Some things might seem a given to those used to running Linux, but as I haven’t done this in many years, it’s always wise to document.

Before You Begin

Continue Reading

FtpWebRequest: More Mainframe Adventures with DotNetCore

Problem:

Our iSeries (mainframe/as400) team has figured out a way to upgrade all our forms and can easily deposit these forms into a folder on the as400.  They are looking for the fastest way to push these forms to our customers.

They ask: Is there a way I can (.NET Core Dev) retrieve their PDF statements from the mainframe (iSeries/AS400) for viewing online? The files are not exposed in Zend (the as400 webserver) but are in a specific “folder” (if you could call it that, the structure of as400 file system is different).

Thought:

Though I actively employ their Zend system to utilize my own API for querying, etc, there was not much interest in this in their team other than knowing how I was calling their programs.  They wanted this SPECIFIC folder to be accessed to retrieve statements.

I tried to propose a HTTP web service via Zend, other HTTP/PHP solutions… but in the end, I explained what I am currently doing:  utilizing my own API to accept an authentication key with parameters (in json using PHP) and convert those parameters to the parameters they need and then calling a program.  The program I call does the logic and I get a response. (In this case I will get a stream).

Solution:

In the end, we decided on a combo of passing parameters via my API (Zend/PHP) and then receiving a file name which I will retrieve via FTP.

I know that I can access files via FTP if given access.  I know I can get to the file(s) they want using FTP and C#.  I do not like it, I insist on many, many security measures but I assure them I can access via FTP and prove it using a FTP client.

It’s decided:  I will use Zend (and PHP) to pass parameters (using post/json) and the response I get will be a file name.  That file, I have to retrieve off of that specific path on the as400.

Problems Particular to Mainframe:

First, I followed the ANSWER to this post:  https://stackoverflow.com/questions/57236179/how-to-download-xlsx-file-using-ftpwebrequest-through-c-sharp-asp-net

I didn’t want to download, as the second half of his post suggested.  I wanted to immediate display (stream) the pdf.  This is how I did it:

[HttpGet]
        public async Task<Stream> GetStatementAsync()
        {
           
            string url = "ftp://1.1.1.1" + "/%2F" + "thisismyfilesharefolder" + "/%2F";
            //TODO:  REPLACE FILE NAME / MAKE DYNAMIC
            string ftpFileName = "MYTESTFILE.PDF";
            var requestUri = new Uri(url + ftpFileName);
            var fileBytes = FtpDownloadBinary(requestUri);

            return new MemoryStream(fileBytes);
        }

        //TODO: MOVE TO FTPHELPER SERVICE
        public byte[] FtpDownloadBinary(Uri fromUri)
        {
            //TODO: REPLACE WITH FTPUSER
            string username = "ASKTHESPAMMER";
            string password = "WHOWANTS10K";

            FtpWebRequest request = (FtpWebRequest)WebRequest.Create(fromUri);
            request.Method = WebRequestMethods.Ftp.DownloadFile;
            request.Credentials = new NetworkCredential(username, password);
            request.UseBinary = true;
            request.KeepAlive = true;
            request.UsePassive = true;
            //TODO: ENABLE FTPSSL ON AS400 TO MAKE THIS POSSIBLE
            //request.EnableSsl = true;

            FtpWebResponse response = (FtpWebResponse)request.GetResponse();

            using (var responseStream = response.GetResponseStream())
            using (var memoryStream = new MemoryStream())
            {
                responseStream.CopyTo(memoryStream);
                return memoryStream.ToArray();
            }
        }

Very raw and in house, our team is making that FTPUSER profile, etc. We just wanted to know if it was POSSIBLE.

Permissions

First error was an obvious authentication error.  The standard profile user I used did not have the same permission I do on FTP.  Simple way to clarify: I did logged on with both users via FTP client (sorry, I still love FileZilla) and with user one I could get in, the other I could not.

Once we established a user with correct permissions, I received another error.

Escapes

“(501) Syntax error in parameters or arguments”

I got past authentication but I kept getting an error somewhere on response.  In truth, the error is simply saying there’s a syntax error in the REQUEST.  The syntax error was a bit hard to find – it ended up being the forward slash in the url.  It looked normal, but the mainframe was denying it.  If you research this, the posts are so old that many of the links are dead.

Quick tutorial for those still dealing with Mainframe / AS400 / iSeries:

If you feed it a string (url) with a slash, it does NOT understand the slash.  You must ESCAPE the slash.

The escape string is this: “/%2F”

in example:  google.com/bugs = google.com/%2Fbugs

for devs: string url = “ftp://1.1.1.1” + “/%2F” + “thisismyfilesharefolder” + “/%2F” + filename

It took forever to find the right escape sequence as so much of the documentation is lost, outdated, etc.

After ftp:// EVERY forward slash must look like this:  /%2F

Notes:

In truth, this is not an “escape” sequence, it’s changing directories forward.  You can also move backward:

https://stackoverflow.com/questions/330155/how-do-you-change-directories-using-ftpwebrequest-net

Unfortunately, so many of the links with information regarding this are dead.

I’m going to tag everything I can on this on hopes that some of you still working with mainframe find it.  Just lending my support as I’m sure I might need yours in the future!

 

Continue Reading

Uploading to Azure Blob Containers: AZCopy 403 Error

The Problem

  • Azure Explorer is uploading too slow
  • AZCopy copy feature will not upload.  Error:  “RESPONSE Status: 403 This request is not authorized to perform this operation using this permission.”

Today, I was trying to upload a directory (1.3gb) of images up for blob storage for public access.   Though I often use Azure Storage Explorer for smaller uploads, it was proving impossible with this large directory.  My connection came to a grinding halt and the files where taking impossibly too long to upload.

So, upon research, I learned that AZCopy is much more efficient with larger data transfers and after success, I found this to be true.  However,  using a direct URL as per the docs was giving me a 403 error, despite me being owner of container and container seeming to have proper permissions.  So I decided to use an SAS key at end of URL (also in docs) and the apply the azcopy copy command.

Below is how I proceeded to get it working.

The Solution

Tools used:

Continue Reading

Notes on Adding .NET Core 2.2 Identity to Existing Project

Even though all my projects are in .NET Core now, I rarely get the opportunity to use Identity because of my work with our backend Legacy system.  Recently, though, I built a very lightweight SEO Management system for one of our sites (that allows a 3rd party to tweak our page titles, meta tags, etc) and wanted to give them user access and roles.

The entire project can be found on GitHub, but below is just a running list of sites I used to get this done, noting all the troubleshooting and stupid little mistakes I did along the way.

Adding Identity to an Already Existing Project

This part was relatively easy and the Microsoft documents provided an easy enough guide.  I believe I had some issues:

CS1902 C# Invalid option for /debug; must be full or pdbonly – with the Data Migrations because I didn’t have EntityFramework installed.   Basically, all errors in this phase were not as presented – they were mostly because I was lacking packages to migrate.

To ensure I had all the right packages installed, i used: Microsoft.AspNetCore.App -Version 2.2.6

Also, in this area, I decided not to put the connection string in appsettings.json, opting instead to use System Environment Variables both in development and in the future on Azure.  I changed my IdentityHostingStartup.cs file to this:

public void Configure(IWebHostBuilder builder)
{
    builder.ConfigureServices((context, services) =>
    {
        services.AddDbContext<DbContext>(options =>
        options.UseSqlServer(System.Environment.GetEnvironmentVariable("SQLAZURECONNSTR_Production")));

Configuring Email

I wanted to test everything first, with no email confirmation.  So, using Microsoft Docs I added this email class.  The only change is again, I prefer System Environment Variables, so my EmailSender class differs:

public class EmailSender : IEmailSender
{
    private string apiKey = System.Environment.GetEnvironmentVariable("SENDGRID_APIKEY");

    public Task SendEmailAsync(string email, string subject, string message)
    {
        return Execute(subject, message, email);
    }

    public Task Execute(string subject, string message, string email)
    {
        var client = new SendGridClient(apiKey);
        var msg = new SendGridMessage()
        {
            From = new EmailAddress("webmaster@mycompany.com", "SEO Management"),
            Subject = subject,
            PlainTextContent = message,
            HtmlContent = message
        };
        msg.AddTo(new EmailAddress(email));

        // Disable click tracking.
        // See https://sendgrid.com/docs/User_Guide/Settings/tracking.html
        msg.SetClickTracking(false, false);

        return client.SendEmailAsync(msg);
    }
}

Once I finished with the Microsoft Docs email setup, I started projected, registered 1 user (under email I want to be super admin).

Customizing User Roles & Seeding Admin

Now, I needed to assign that user the role of super admin and also create other user roles.  I used the following answer to get started added both methods to Startup.cs in the Configure method.

https://stackoverflow.com/questions/55994082/how-to-create-roles-in-asp-net-core-2-2-and-assign-them-to-users?answertab=votes#tab-top

            app.UseMvc(routes =>
    {
        routes.MapRoute(
            name: "default",
            template: "{controller=Home}/{action=Index}/{id?}");
    });
 
 
    // this is just to seed other admin role, run once
    //CreateAdminRole(serviceProvider).Wait();

}
 
// this is just to seed other admin role, run once
private async Task CreateAdminRole(IServiceProvider serviceProvider)
{
    var RoleManager = serviceProvider.GetRequiredService<RoleManager<IdentityRole>>();
    var UserManager = serviceProvider.GetRequiredService<UserManager<ApplicationUser>>();
 
    IdentityResult roleResult;
 
    var roleCheck = await RoleManager.RoleExistsAsync("Admin");
    if (!roleCheck)
    {
        roleResult = await RoleManager.CreateAsync(new IdentityRole("Admin"));
    }
 
    ApplicationUser user = await UserManager.FindByEmailAsync("myemail@mycompany.com");
    await UserManager.AddToRoleAsync(user, "Admin");
}
 

After following that guide, the one error I kept getting here was:  No service for type ‘Microsoft.AspNetCore.Identity.RoleManager’.   

and another:  Unable to resolve service for type ‘Microsoft.AspNetCore.Identity.IRoleStore`1[Microsoft.AspNetCore.Identity.IdentityRole]’ while attempting to activate ‘Microsoft.AspNetCore.Identity.RoleManager`1[Microsoft.AspNetCore.Identity.IdentityRole]’.

The first was because I simply forgot to .AddRoles to the configuration of the DefaultIdentity.  The second error is that ORDER MATTERS.  AddRoles goes before AddEntityFramework.

Both resolved with this:

services.AddDefaultIdentity<ApplicationUser>()
            .AddRoles<IdentityRole>()
            .AddEntityFrameworkStores<ApplicationDbContext>();

Finishing with the guide, I discovered another page that needed changing: _ManageNav which also had a SignIn @inject (like _LoginPartial) that needed to be changed:

@inject SignInManager<ApplicationUser> SignInManager

And then: InvalidOperationException: Unable to resolve service for type ‘Microsoft.AspNetCore.Identity.UserManager`1[

Which, again, is all pointing to the use of IdentityUser instead of your new ApplicationUser.  You’ll need to go through each of your Razor pages and change IdentityUser to ApplicationUser.

Which will lead to you scaffolding the views (if you hadn’t already) …

Scaffolding UI for Access to Razor Pages & Views

Now that the set up was working, I wanted a better look at my pages and views.  By default, in Core 2.1 the UI comes in a prebuilt package, but you can easily scaffold it to view and change as you like: ASP.NET Core 2.2 – Scaffold Identity UI.  I even did this over (ie, a 2nd time) over my first steps and it recognized all the previous code and just scaffolded nicely, no harm done.

Adding User Roles

At this point, any user can register, but they can not log in unless they confirm their email.  Once they DO confirm their email, they can log in but have no access to any of the pages because they have yet to be assigned a role.  The administrator must do this before they can continue.

Using the same method that I used to create the seed admin, I added a method to startup to seed the user roles:

// this is just to seed user roles, run once
private async Task CreateUserRoles(IServiceProvider serviceProvider)
{
    var RoleManager = serviceProvider.GetRequiredService<RoleManager<IdentityRole>>();

    IdentityResult roleResult;

    var roleCheck = await RoleManager.RoleExistsAsync("SEOAdmin");
    if (!roleCheck)
    {
        roleResult = await RoleManager.CreateAsync(new IdentityRole("SEOAdmin"));
    }

    roleCheck = await RoleManager.RoleExistsAsync("SEOManager");
    if (!roleCheck)
    {
        roleResult = await RoleManager.CreateAsync(new IdentityRole("SEOManager"));
    }

}

Then, added

CreateUserRoles(serviceProvider).Wait();

To Configure method in Startup.cs, ran once and the roles were set.

Assigning Roles to Users

Then, I wanted something I could see existing users and decide their role before they got access.  The end result looks like this:

To do this, I added a ManageUsersController and retrieved Users and Roles so that I could assign them a role.  I created a the User Role View Model and User View Model to reflect the drop downs and added a corresponding view.

That was the last step to getting this little SEO backend together.

All packaged together, the entire project can all be seen on Github.

 

Continue Reading

Site 24×7 and JSONPath Example

We had an issue a couple of days ago where two points of failure caused some downtime and so I spent most of today revisiting all my monitors to give us as much warning as possible if one of our services fails.

One particular API I set up lives on the as400 (iSeries) and operates off of it’s ZendCore server in PHP.  I quickly scripted a page to verify:  connection between ZendCore PHP and db2, that the connection to that endpoint was in fact SSL, and on what port.

The PHP code is very simple and returns a JSON string that looks like this:

{
    "AS400CONNECTION": true,
    "SSL": true,
    "PORT": "633"
}

The slight problem today was figuring out how to get Site 24×7 to not just verify the link, but check the JSON values and verify it is what I need.  If not, then send the alert.  In this case, I want to verify AS400CONNECTION is true and SSL is true.

Site 24×7 suggests using a REST API monitor and then asserting the value in JSONPath.  I was completely new to this and finding a good clean example was a bit tough, hopefully this saves someone some time.  Here’s how I setup the JSONPath expression:

$[?(@.AS400CONNECTION==true)]
$[?(@.SSL==true)]

For a full screenshot of the setup I did to make this work click here.

 

 

 

Continue Reading

Making Session Variables Work in .NET Core

The Problem

As noted, I work with Legacy and often have to bring in variables from the API that must be sustained across session.. (and I’m sure there might be a better way, comment and advise!).  Where I am at now, is I query the API and bring in the variables, but how do I keep from calling these over and over?  The old solution was session variables and so, that’s where I am at.

When I started to do this on Core, the most helpful article was this (and it’s in my comments):

https://adamstorr.azurewebsites.net/blog/are-you-registering-ihttpcontextaccessor-correctly

He leads you through the basic setup of a HttpContext helper class (that I still use today) and how to configure the startup..  Today, though, I came across a problem: I was able to Set session variables, but the Get was pulling null.

The Reason

Order.  Yes, you’ll see 1000 stackflow responses about order in Configure (and I was careful to do this in that method), but now in ConfigureServices (contrary to the example, as I am now using Core 2.2?), order again comes into play:

public void ConfigureServices(IServiceCollection services)
{
    //this MUST be before add mvc or session returns null
    services.AddSingleton<IHttpContextAccessor, HttpContextAccessor>();
    services.AddMvc();

Identify It

How does the error present itself?  Debugging looks great, queries to API fine, setting session (cookies) fine,  result unexpected, but..  no errors.  Trace your Get.  My Session.GetString was pulling null.

Solution

Switch order  in ConfigureServices and all was fine.

 

 

 

Continue Reading