Category Archives: C#

Quick Tangent: It’s Probably for the Best

So, it’s been a while since I talked about indoor navigation. It’s one of those things that I always come back to, especially since that idea for the ghost game always comes back to me now and again. After a conversation with a hardware grad student in a PhD program, I got excited about the idea again and went looking once more for a software solution. As it turns out, Microsoft wants in on the action. After playing with it for a while, though, there’s only one problem: much like other indoor navigation solutions, it doesn’t work exactly.

In my apartment several stories up and which occupies only one floor, I will walk several feet. Then it will suddenly prompt me, asking me which floor I’m headed to. Apparently, it thinks that I’m in an elevator or on an escalator.

With all the difficulties amassed between AR and navigation, it’s no wonder that Project Tango was closed by Google. And it’s no wonder that this Microsoft navigation project apparently hasn’t been updated for a year now. After all, AR and indoor navigation are tough subjects to tackle.

So, it’s refreshing to hear that Microsoft might be rethinking some of its past approaches. After having experimented with their earlier iterations of Windows IoT, I found it an interesting foray for Microsoft. However, I didn’t really believe that it’d be adopted by manufacturers and (especially) developers. It seems that Microsoft has had the same realization recently, and it’s now pursuing a new project to revamp their IoT (and mobile, to some degree) portfolio called Azure Sphere. Now, this initiative could maybe breathe new life into some of that confused tech. If somebody out there creates a kit for Azure Sphere, I’m a taker. I’m looking at you, Adafruit!

Advertisements

Resist Bad Data, Part 2: How to Filter Incomplete XHTML Entities and Encodings

So, as stated in the piece before, dealing with poorly formed XML is a necessary (though infuriating) part of the day for some people, including me. In that article, I described which regular expression to use (with Perl) in order to cleanse your files of such garbage. Which is fine, especially when you’re scripting…but these days, I’m fairly sure that the clear majority of systems are less about batch jobs and more are programming-oriented (distributed architectures, microservices, etc.). So, obviously, it’d be nice to have a solution that could be part of a platform with a more robust programming language.

So, how do we do it in Java? Easy enough. We can just make use of the same regular expression mentioned last time, since Java has a fairly straightforward approach that mimics other languages (like Perl):

—————-

java.nio.file.Path wiki_path = java.nio.file.Paths.get("C:/onix_test_data/test_files", "test_file_ONIX.xml");

java.nio.charset.Charset charset = java.nio.charset.Charset.forName("ISO-8859-1");

try {

StringBuilder FileContents = new StringBuilder();

List lines = java.nio.file.Files.readAllLines(wiki_path, charset);

for (String line : lines) {
     FileContents.append(line + "\n");
}

String sAllFileContents = FileContents.toString();

String sUpdatedFileContents = sAllFileContents.replaceAll("(&#?x?[A-Za-z0-9]+;)|&#\\d*", "$1");

try(java.io.PrintWriter out = new java.io.PrintWriter( "C:/onix_test_data/output/test_file_ONIX.filtered.xml" ) ) {
     out.println( sUpdatedFileContents );
}
} catch (IOException e) {
     System.out.println(e);
}

—————-

And what about C#? Well, as I’ve pointed out again and again, the .NET platform isn’t exactly a source of inspiration when it comes to handling XML. And, of course, it has to be different when it comes to everything, including regular expressions. So, as my personal recommendation, I would make use of regular expressions for pattern matching and make use of the callbacks for actual filtering:

—————-

var sAlteredOutput = Regex.Replace(sOnixTestXml, @"&#?x?[A-Za-z0-9]*;?", FilterMatcher, RegexOptions.Singleline);

...

static public string FilterMatcher(Match m)
{

string sResult = "";

try
{
    if (m.Value == "&#")
        sResult = "";
    else if (m.Value.StartsWith("&#") && (m.Value.Length > 2))
    {
        int nEncodingVal = 0;
        string sInsideEncoding = m.Value.Substring(2);

        if (m.Value.EndsWith(";"))
            sResult = m.Value;
        else if (!Int32.TryParse(sInsideEncoding, out nEncodingVal))
        {
            int nIdx;
            for (nIdx = 0; nIdx < sInsideEncoding.ToCharArray().Length; ++nIdx)
            {
                char cTemp = sInsideEncoding.ToCharArray()[nIdx];

                if (!(cTemp == 'x') && !Char.IsDigit(cTemp)) break;
            }

            if (nIdx == sInsideEncoding.ToCharArray().Length)
                Result = m.Value;
            else
                sResult = m.Value.Substring(nIdx + 2);
        }
        else
            sResult = "";
    }
    else if (m.Value.StartsWith("&"))
        sResult = m.Value;

}
catch (Exception ex)
{
    sResult = m.Value;
}

return sResult;
}

}

—————-

In fact, I’d rather use this solution with callbacks since I love the idea of having more programmatic control. But don’t tell Microsoft that. 😛

And there you go. Two solutions to help you work around the various forms of ineptitude of data suppliers…of which there seems to be no end in sight.

Red Shirt Tour NYC

A few weeks ago, Microsoft VP Scott Guthrie stopped in NYC as part of his promotional tour for Azure’s cloud services. Normally, I don’t really care for these long informercials, but I decided to go in this case for two reasons. One, it took place in Cooper Union’s Great Hall, which was something historic I had always wanted to check out. Two, even though I’ve played with Azure’s offerings on occasion, I was curious what Guthrie would highlight in his presentation, especially after friends and colleagues had talked up Azure in the last couple of years. So, I went, and I was surprisingly glad that I did.

A few years ago, when I learned of some of Microsoft’s ambitions in the cloud space, I bought some MSFT shares and thought that they might catch up with Amazon in the cloud space. Impatiently, after a year, I began to have my doubts, and I sold off the shares. If you look at its latest price, that was clearly a mistake. I should have held onto them, and while listening to Guthrie’s presentation, it became painfully obvious as to why. Of course, he talked about the inherent power of Azure, with its various data centers around the world. (Which were all shown in a dramatic video seemingly directed by Michael Bay, with an intensely dramatic score blaring in the background.) However, it was the maturity of the platform, with its various tools and considerations for the user that was impressive.

Even though I’ve only dabbled with cloud platforms, I especially appreciate their raw power and penchant for structure. Even when you create apps that are meant to be deployed for the cloud, it heavily reinforces structure in their templates for developers. For example, if you build a web app within Visual Studio destined for Azure, it refers to queues in the project template by default (which are automatically available in Azure). You might question its need to reinforce such a feature through templates, thinking that the usage of queues in a web app as obvious. “What vital web service that receives a POST wouldn’t automatically queue that request, since the resources to fulfill the request might be temporary unavailable? Who wouldn’t do that?” Oh, you’d be surprised. Let’s just say that such bug-ridden deployments are not unheard of.

First, Guthrie talked about the typical use cases of cloud offerings, like creating and deploying a serverless app with ease or creating a container and deploying it with Kubernetes. Even when he talked about deploying apps to specific data centers, those features were interesting but expected. (I still have mixed feelings about the Azure Portal dashboard, but since that’s an argument about interfaces, that’s an entirely different subject.)

However, I was more impressed when he started talking about the other free tools offered by the platform that were supplemental. For one, its inherent monitoring system was akin to Splunk, and it could be used to monitor and query your entire setup (apps, databases, hosted servers, etc.), and you could customize your system instances with networking rules (like lock port 54545 every morning). Next, the number of devops options seemed more diverse than I remembered (with build options including Maven). There was even a tool which would suggest how you could reduce your expenses, like by consolidating servers and minimizing resources (storage, # of dedicated CPUs, etc.). After watching some examples of machine learning and image recognition, my only regret was that I had no reason to use them in my own projects.

After several years of concentrated effort, they had created an impressive, mature cloud platform that definitely could give Amazon a run for its money. I only had two points of contention. One, this promotional tour was somewhat eponymously named The Red Shirt Tour, since I was told by staff that Guthrie has a certain love for them. Aside from a terrible name for the tour, I would say ditch the symbolic shirt, since that will forever more belong to Jobs. Pick a hat, and in order to reinforce Azure, go for a blue beret instead.

Two, they served Subway for lunch. To which the answer is always no.

Yes, it was free. However, just like if you were offered torture for free, you shouldn’t accept it.

Aside from that, though, I walked away impressed. Nicely done, Guthrie and MS!

And While I’m Asking For Things

Speaking of requests, I have another quick one. My department has used the Oracle.ManagedDataAccess.dll for quite a while now in our C# (i.e., .NET) deployments, and even though we’ve had some performance issues at times, we’ve created some workarounds and eventually achieved harmony with this miscreant spawn of Oracle (who has been known to be developer-unfriendly). However, we came across a new one the other day:

Value cannot be null.
Parameter name: byteArray
Server stack trace:
at System.BitConverter.ToString(Byte[] value, Int32 startIndex, Int32 length)
at OracleInternal.TTC.TTCLob.GetLobIdString(Byte[] lobLocator)
at OracleInternal.ServiceObjects.OracleDataReaderImpl.CollectTempLOBsToBeFreed(Int32 rowNumber)
at Oracle.ManagedDataAccess.Client.OracleDataReader.ProcessAnyTempLOBs(Int32 rowNumber)
at Oracle.ManagedDataAccess.Client.OracleDataReader.Read()

Basically, if you have a null value in the CLOB column of an Oracle table, you might get this error when you attempt to access that row in the result set. After trying the most recent version of the library (as recommended by Oracle), it became obvious that they still haven’t fixed it. Strangely, the error seems to occur randomly, since it won’t always happen when running the executable against the same data. After spending some time in an effort to diagnose the error, I eventually capitulated, accepted Oracle’s foibles, and simply set all CLOB columns to EMPTY_CLOB(). Problem solved.

But for posterity and all those after me, I implore you, Oracle: can you address this simple problem in your library? I’d ask you to open-source that library, but we all know that’s not gonna happen. So, please go ahead and fix the bug for us. It’d probably take you a whole 15 minutes, the same amount of time that Ellison needs to check the rigging on one of his catamarans. We’d all appreciate it!

.NET and XML: Mortal Enemies

In the past, I’ve had my issues when dealing with the compatibility between C# and XML, so much so that I’ve been inspired to write prose. At this point, I feel that it’s my duty for posterity to document the various instances where Microsoft drops the proverbial XML ball:

  1. As stated before, DTD validation of XML files (with the C# XmlReader) is broken. And when I say broken, I mean that it performs as well as a thirty-year-old VCR pulled from the ocean and then placed in a vat of hydrochloric acid. Now plug that VCR in. You’re going to get the same results.
  2. If you’re hoping to create a XSD from available XML or classes with the Visual Studio Tool (i.e., XSDT), good luck. For one, it doesn’t allow you to specify any complex rules. Also, if you’re generating a XSD schema from classes, you should know that it doesn’t work with Nullable types or any Dictionary properties, since they couldn’t budget it into their schedule.
  3. “Hey, I have a container that’s set to null. So I’ll tell C# not to mention that tag at all upon serialization, since it’s just wasted space and since I might not want that tag visible in certain cases.” It is possible…but under specific circumstances. If you plan on serializing your class as is and if you plan on not using any C# attributes on your properties, maybe. Otherwise? Nope. C# says “No soup for you!”
  4. If you want to serialize an array of strings, you get a free bonus upon serialization: all of the tags will be prefixed with a namespace (i.e., the namespace for array serialization). If you don’t want that ugly prefix, just create a wrapper container for a string array. Now do this for every literal type you can think of.
  5. Let’s say that you have a XML doc with a string that is a HTTP link. You now want to parse that XML and read that value. Of course, if that link contains a query string, all of those ampersands are already converted into instances of “&amp;“, in order to be compatible with XML standards. “But I want my data to be read incorrectly!” Well, you’re just in luck, friend! Because when you use XmlDocument.Load() to deserialize that XML, that query string will contain “amp;” and not “&amp;“, essentially turning the link into garbage. Huzzah!

These are only a few instances, but there’s more. (Don’t even get me started on deserializing payloads from a ASP.NET web service.) If anyone out there has their own experiences to add, I’d love to hear them.