Calling Roslyn from Net Framework and Net Core

It never ceases to amaze me how people can write a huge API and never bother to document how to use it. But, it’s been that way for as long as I can remember, going back 35 years. In my latest adventures, I’ve been trying to compile, link, and run C# code dynamically using Roslyn for Piggy, my transformational system. If you’ve ever used Roslyn in C#, you’ve probably discovered that it can be such a pain in the arse to use because Microsoft gives doc for the APIdoes give some tutorials, but I can’t find a simple example for compiling, linking, and running C#. I don’t need to know all the details yet, just a starting point framework. Unfortunately, the solution is quite sensitive to whether you use Net Core or Net Framework.

Continue reading

Posted in Tip

MSBuild rules for Antlr4 grammars using Antlr4BuildTasks

In order to better support Piggy, which uses Antlr4, I’ve added a NuGet package called Antlr4BuildTasks. This package is a pared-down derivative of the excellent work of Sam Hartwell Antlr4cs code generator package, and includes just the rules and code needed to do builds in MSBuild, Dotnet, or Visual Studio 2017 IDE–just no Antlr4 tool itself. This package decouples the build rules from the Antlr4 tool and runtime, so you can build Antlr programs using the latest Java-based Antlr tool and runtime release. To use this package, make a reference to this package as if you would to any NuGet package; make sure to also reference the Antlr4.Runtime.Standard package, install Java, the Java-based Antler tool, and set JAVA_HOME and Antlr4ToolPath. The tool works with Net Core, Net Framework, or Net Standard code, and on Windows or Linux.

Continue reading

Posted in Tip

This little piggy is not at the market yet

With a bit of hacking for the last month or two, and I can finally see that I am making progress on Piggy, a new kind of p/invoke generator. Some might say “Why in the world are you wasting time writing a p/invoke generator? Aren’t there tools already that do this?” Well, yeah, there are other generators, but they all…how should I say…suck! I need a p/invoke generator for Campy, a compiler and runtime for C# for GPUs, which I am still working on, but had to place on the back burner to work on this. Campy uses LLVM and CUDA. Because these libraries are large and constantly changing, I have to have an automated way of handling new releases.

Continue reading

Posted in Tip

Re-inventing the p/invoke generator

If you’ve been programming in C# for a while, at some point you found yourself needing to call C libraries. It isn’t often, but when you have to do it, it’s like pulling teeth. One option is to set up a C++/CLI interface; the other is a p/invoke interface to a DLL containing the C code. It’s relatively easy to set up a p/invoke interface in your C# code for the C code, which you export with a DLL–if you only need to call a few C functions. But, if the API is large, you stare at the code for a while, deciding whether it is really worth writing out all the declarations you need to make the calls. Many people throw caution to the wind, write packages for large, popular C APIs so you don’t have to, which you can find on the Nuget website. One example is ManagedCuda, an API for CUDA programming from C#. Unfortunately, people get tired of trying to keep these packages up to date, and so these packages become obsolete. Another approach is through automatic means, whereby a tool reads the C++ headers (or DLL) and output the decls you call. A p/invoke generator reads C header files and outputs C# code with the p/invoke declarations that you can include in your code. These tools sometimes work, but often they don’t.

This blog entry is a “heads up” note about my thoughts for a new type of p/invoke generator. Continue reading

Porting Re2/j to C#

For these last few weeks, I’ve been trying to grapple with the problem of p/invoke–the nasty but must-use feature in C#. While one could write these declarations out by hand, some libraries are too large, and change too often, so people use p/invoke generators, like SWIG. However, the is no generator that is easy to use or generates 100% correct C# declarations. So, as every software developer does, so I go to re-invent the wheel.

Continue reading

Posted in Tip

Useful websites for algorithm designers and programmers

I’ve been lax the last six months on my blog, working instead on Campy (a C#/GPU programming language extension). Now that that is slightly under control, time to get back to the blog. And, by the way, the whole reason for Campy is to implement popular algorithms to run on a GPU, I thought I’d take some time to review what information is available on the internet on algorithms. The following is a list I’ve been working on for a few months. It is by no means a complete list. But, I hope it covers some of the more popular sites. The entries are not in any particular order. Note, this list does not include parallel algorithms, which will be a post unto itself, nor the seemingly required AI algorithms you must have nowadays.

Continue reading

Posted in Tip

Windows recover tools for an idiot

I’m going to tell a story that probably many of you can relate to.

Last week, I experienced a slow motion catastrophic failure in Windows. It surprise me because ever since Windows 7, I’ve had pretty good luck in rolling forward with the updates and upgrades of the OS. But, that changed when I moved to the Windows 10 Creator Update.

As Microsoft posted in its website, “Why wait? Download Creator’s Update now.” So I did.

Unfortunately, it started with problems with an old NVIDIA Ti 470 graphics card. More importantly, I couldn’t run a Samsung Android virtual machine with VirtualBox due to a problem in a VirtualBox network adapter driver.

I tried the usual: reinstalling VBox, some registry edits of some VBox settings, and sanity checks on the OS and disk drives. But, no matter what I tried, I couldn’t get VBox working.

Biting the bullet, I decided on increasingly drastic measures: a roll back to a previous restore point; a Windows 10 Reset (keeping user files). While the VBox problem was fixed, Windows broke in the process: I couldn’t use Windows 10 Start and type in “cmd” to run a shell.

In desperation, I decided to try a full Windows 10 Reset (removing all user files), but stopped immediately when Windows prompted whether I wanted to delete all data from all drivers in the machine! Really?? Being old school, I decided just to do a format/install fresh copy of the OS instead. I know, so “old school”.

In the end, I was able to get the PC back up and restore my files. But, I learned a few things about the tools MS offers to fix a Windows.

System File Integrity

SFC — System File Checker

Verifies the validity of Windows system files.

C:\Windows\system32>sfc /help

Microsoft (R) Windows (R) Resource Checker Version 6.0
Copyright (C) Microsoft Corporation. All rights reserved.

Scans the integrity of all protected system files and replaces incorrect versions with
correct Microsoft versions.

    [/OFFWINDIR=<offline windows directory> /OFFBOOTDIR=<offline boot directory>]

/SCANNOW        Scans integrity of all protected system files and repairs files with
                problems when possible.
/VERIFYONLY     Scans integrity of all protected system files. No repair operation is
/SCANFILE       Scans integrity of the referenced file, repairs file if problems are
                identified. Specify full path <file>
/VERIFYFILE     Verifies the integrity of the file with full path <file>.  No repair
                operation is performed.
/OFFBOOTDIR     For offline repair specify the location of the offline boot directory
/OFFWINDIR      For offline repair specify the location of the offline windows directory


        sfc /SCANNOW
        sfc /VERIFYFILE=c:\windows\system32\kernel32.dll
        sfc /SCANFILE=d:\windows\system32\kernel32.dll /OFFBOOTDIR=d:\ /OFFWINDIR=d:\windows
        sfc /VERIFYONLY


Basic File System Integrity

Chkdsk — Checks a disk and displays a status report

This tool checks the validity of a file system. It used to check for bad sectors in a drive, but that is no longer performed by the tool.

C:\Windows\system32>chkdsk /?
Checks a disk and displays a status report.

CHKDSK [volume[[path]filename]]] [/F] [/V] [/R] [/X] [/I] [/C] [/L[:size]] [/B] [/scan] [/spotfix]

  volume              Specifies the drive letter (followed by a colon),
                      mount point, or volume name.
  filename            FAT/FAT32 only: Specifies the files to check for
  /F                  Fixes errors on the disk.
  /V                  On FAT/FAT32: Displays the full path and name of every
                      file on the disk.
                      On NTFS: Displays cleanup messages if any.
  /R                  Locates bad sectors and recovers readable information
                      (implies /F, when /scan not specified).
  /L:size             NTFS only:  Changes the log file size to the specified
                      number of kilobytes.  If size is not specified, displays
                      current size.
  /X                  Forces the volume to dismount first if necessary.
                      All opened handles to the volume would then be invalid
                      (implies /F).
  /I                  NTFS only: Performs a less vigorous check of index
  /C                  NTFS only: Skips checking of cycles within the folder
  /B                  NTFS only: Re-evaluates bad clusters on the volume
                      (implies /R)
  /scan               NTFS only: Runs an online scan on the volume
  /forceofflinefix    NTFS only: (Must be used with "/scan")
                      Bypass all online repair; all defects found
                      are queued for offline repair (i.e. "chkdsk /spotfix").
  /perf               NTFS only: (Must be used with "/scan")
                      Uses more system resources to complete a scan as fast as
                      possible. This may have a negative performance impact on
                      other tasks running on the system.
  /spotfix            NTFS only: Runs spot fixing on the volume
  /sdcleanup          NTFS only: Garbage collect unneeded security descriptor
                      data (implies /F).
  /offlinescanandfix  Runs an offline scan and fix on the volume.
  /freeorphanedchains FAT/FAT32/exFAT only: Frees any orphaned cluster chains
                      instead of recovering their contents.
  /markclean          FAT/FAT32/exFAT only: Marks the volume clean if no
                      corruption was detected, even if /F was not specified.

The /I or /C switch reduces the amount of time required to run Chkdsk by
skipping certain checks of the volume.


Bad Disk Drive Checks

There are a number of free tools which check for bad sectors on a hard disk drive (HDD). You should not use these tools on solid state drives (SSD) as these checks shorten the life of the SSD.

Many of the tools are manufacturer-specific programs. For example, Seatools is specific for Seagate drives. Sandisk SSD Dashboard is specific for Sandisk. Macrorit Disk Scanner seems pretty good tool with a colorful GUI to boot, and is not manufacturer specific.

Windows trouble shooter

Windows provides some trouble shooting programs to help detect and fix problems. You can operate them from the GUI, or the command line.

msdt.exe /id name-of-check

Available checks:


DISM — Deployment Image Servicing and Management tool

DISM enumerates, installs, uninstalls, configures, and updates features
and packages in Windows images.

C:\Windows\system32>dism /help

Deployment Image Servicing and Management tool
Version: 10.0.15063.0

DISM.exe [dism_options] {Imaging_command} [<Imaging_arguments>]
DISM.exe {/Image:<path_to_offline_image> | /Online} [dism_options]
         {servicing_command} [<servicing_arguments>]


  DISM enumerates, installs, uninstalls, configures, and updates features
  and packages in Windows images. The commands that are available depend
  on the image being serviced and whether the image is offline or running.


  /Split-Image            - Splits an existing .wim or .ffu file into multiple
                            read-only split WIM/FFU files.
  /Apply-Image            - Applies an image.
  /Get-MountedImageInfo   - Displays information about mounted WIM and VHD
  /Get-ImageInfo          - Displays information about images in a WIM or VHD
  /Commit-Image           - Saves changes to a mounted WIM or VHD image.
  /Unmount-Image          - Unmounts a mounted WIM or VHD image.
  /Mount-Image            - Mounts an image from a WIM or VHD file.
  /Remount-Image          - Recovers an orphaned image mount directory.
  /Cleanup-Mountpoints    - Deletes resources associated with corrupted
                            mounted images.

  /Apply-CustomDataImage  - Dehydrates files contained in the custom data image.
  /Capture-CustomImage    - Captures customizations into a delta WIM file on a
                            WIMBoot system. Captured directories include all
                            subfolders and data.
  /Get-WIMBootEntry       - Displays WIMBoot configuration entries for the
                            specified disk volume.
  /Update-WIMBootEntry    - Updates WIMBoot configuration entry for the
                            specified disk volume.
  /List-Image             - Displays a list of the files and folders in a
                            specified image.
  /Delete-Image           - Deletes the specified volume image from a WIM file
                            that has multiple volume images.
  /Export-Image           - Exports a copy of the specified image to another
  /Append-Image           - Adds another image to a WIM file.
  /Capture-Image          - Captures an image of a drive into a new WIM file.
                            Captured directories include all subfolders and
  /Get-MountedWimInfo     - Displays information about mounted WIM images.
  /Get-WimInfo            - Displays information about images in a WIM file.
  /Commit-Wim             - Saves changes to a mounted WIM image.
  /Unmount-Wim            - Unmounts a mounted WIM image.
  /Mount-Wim              - Mounts an image from a WIM file.
  /Remount-Wim            - Recovers an orphaned WIM mount directory.
  /Cleanup-Wim            - Deletes resources associated with mounted WIM
                            images that are corrupted.


  /Online                 - Targets the running operating system.
  /Image                  - Specifies the path to the root directory of an
                            offline Windows image.


  /English                - Displays command line output in English.
  /Format                 - Specifies the report output format.
  /WinDir                 - Specifies the path to the Windows directory.
  /SysDriveDir            - Specifies the path to the system-loader file named
  /LogPath                - Specifies the logfile path.
  /LogLevel               - Specifies the output level shown in the log (1-4).
  /NoRestart              - Suppresses automatic reboots and reboot prompts.
  /Quiet                  - Suppresses all output except for error messages.
  /ScratchDir             - Specifies the path to a scratch directory.

For more information about these DISM options and their arguments, specify an
option immediately before /?.

    DISM.exe /Mount-Wim /?
    DISM.exe /ScratchDir /?
    DISM.exe /Image:C:\test\offline /?
    DISM.exe /Online /?


Additional examples:

dism /online /cleanup-image /checkhealth
dism /online /cleanup-image /scanhealth

Windows Driver Verifier

Verifier.exe is a tool to check drivers installed on your system. It is a Desktop GUI application. Be very careful with this program: you can easily trash your system using the tool–from the voice of experience! Make sure to create a backup before proceeding. A good intro is here.

Recreate a Profile

Sometimes recreating a profile may solve your problems. See

How to Start Windows in Safe Mode with Command Prompt

Once in a while, you may need to go into “Safe Mode” when booting Windows. If the PC can boot, try this at the login screen: hold down the Shift key and click on the power button and then click on Restart. If the PC cannot boot, you must boot from a recovery disk. Make sure you do that before trouble hits! Plug in a flash drive, open Control Panel’s Recovery tool, then click Create a recovery drive.


Posted in Tip