Another Day, Another Session with Redis

So, after the last time with Redis, my team and I still felt uninformed about the features that Redis offers with its Enterprise version. And so, we were told that another session was coming up, one that would perhaps give us the answers we sought. Alas, that wasn’t really the case…but, hey, Redis definitely knows how to play the part of being a host. In my book, you get points for that.

This time, the event was held at Convene, which is uptown from Galvanize. In terms of accommodations, this hosting space was definitely one of the best that I’ve ever been to, especially in terms of food and a view:

And you have to appreciate any place that takes its coffee and yogurt seriously:

In the end, though, it seemed to be a similar presentation to the one a few months ago. We already knew about the HA functionality and regional synchronization that was available through the Enterprise version, but we were looking to possibly see it in action, along with the other bells and whistles. Oh well. Maybe next time…especially if it’s held again at Convene!

Advertisements

Quick Tangent: It’s Probably for the Best

So, it’s been a while since I talked about indoor navigation. It’s one of those things that I always come back to, especially since that idea for the ghost game always comes back to me now and again. After a conversation with a hardware grad student in a PhD program, I got excited about the idea again and went looking once more for a software solution. As it turns out, Microsoft wants in on the action. After playing with it for a while, though, there’s only one problem: much like other indoor navigation solutions, it doesn’t work exactly.

In my apartment several stories up and which occupies only one floor, I will walk several feet. Then it will suddenly prompt me, asking me which floor I’m headed to. Apparently, it thinks that I’m in an elevator or on an escalator.

With all the difficulties amassed between AR and navigation, it’s no wonder that Project Tango was closed by Google. And it’s no wonder that this Microsoft navigation project apparently hasn’t been updated for a year now. After all, AR and indoor navigation are tough subjects to tackle.

So, it’s refreshing to hear that Microsoft might be rethinking some of its past approaches. After having experimented with their earlier iterations of Windows IoT, I found it an interesting foray for Microsoft. However, I didn’t really believe that it’d be adopted by manufacturers and (especially) developers. It seems that Microsoft has had the same realization recently, and it’s now pursuing a new project to revamp their IoT (and mobile, to some degree) portfolio called Azure Sphere. Now, this initiative could maybe breathe new life into some of that confused tech. If somebody out there creates a kit for Azure Sphere, I’m a taker. I’m looking at you, Adafruit!

Quick Tangent: There’s a Stranger in My House

Usually, I don’t find OS updates particularly exciting, and I generally favor waiting for everyone else to take the hit in becoming a first adopter. However, I found this particular note about Android P very interesting, enough so that I might sit in the front row of class and raise my hand:

Today’s preview includes the following new APIs and features (but you can expect much more; this is just the first preview, after all): Display cutout support; HDR VP9 Video, HEIF image compression, and Media APIs; HEIF (heic) images encoding has been added to the platform; multi-camera API; ImageDecoder for bitmaps and drawables; Improved messaging notifications; Data cost sensitivity in JobScheduler; indoor positioning with Wi-Fi RTT: Platform support for the IEEE 802.11mc WiFi protocol — also known as WiFi Round-Trip-Time (RTT) — lets you take advantage of indoor positioning in your apps.

INDOOR POSITIONING!? Well, that might change the whole situation. Maybe I should dust off the old code, crack some knuckles, and get to work!

Galvanize This: Hanging Out with Redis

So, I attended a Redis workshop a few days ago, at the New York “campus” (which is a buzzy, DB-esque industry word that I loathe) of Galvanize. Even though I usually don’t go for these meetup/workspace type of places, I’ll admit that this one was rather pleasant. Unlike other places, it did a good job of finding that fine line between casual and professional. For example, no beanbags anywhere. Because as much as I love beanbags myself (i.e., I have two at home), we can’t look at your screen together unless I get on my knees or I crawl onto the beanbag with you…which might be uncomfortable in many ways for the both of us. Plus, the space had reliable WiFi for most of the time I was there, unlike some other places.

Even though I’ve already been dealing with Redis at work for a little while now and generally impressed with its performance, it never hurts to try and learn something from the masters. (Unfortunately, AntiRez himself did not leave Sicily and fly over to teach us.) I was curious how they were going to showcase the tech and if we were going to just sit there and watch, when they instructed us to download Docker. As it turned out, we were going to learn the lesson via containers with Jupyter Notebooks, which I had never heard of before. And since I yearn for the era of interactive documentation, I couldn’t have been happier. (On a side note, I only recently learned about KataCoda, which I love just as much, if not more.)

Even though the second half of the day was your familiar salespitch for Redis Enterprise and Redis Cloud (which did seem to be an appealing purchase), the first half of the day was when they taught about the product itself. For the most part, it wasn’t anything new to me, aside from the occasional bit of trivia. (Lua is the language used by the Redis CLI? Huh. It’s come a long way since being just the scripting language for WoW skins.) I did learn a few tidbits about the data structures (like the existence of HyperLogLog), but since I use Spring Caching in our microservices, we don’t pay that much attention to them.

Instead, it was interesting to learn about the features of Redis that we weren’t even leveraging yet at work. For example, you can write your own extensions to Redis using C. Which I’d be tempted to do just because, since I miss writing in C…Also, it was interesting to learn about the various modules that were already available for Redis, with functionality ranging from machine learning to bloom filters. And that’s when I recognized a pattern seen before. Much like every other tech company, there seems to be the desire to get into whatever is hot, to survive as a company by being more horizontal. However, I would implore Redis to be careful and to never neglect your core mission. I, for one, don’t really need machine learning, but I’d like Redis to work with Spring (i.e., Pivotal) to further develop the Spring Data Redis layer and make it configurable, so I can easily direct reads to slave nodes. I need that, not machine learning. So, even though I didn’t learn a great deal about Redis by attending, I got to see the general direction of the company. In that way, I’d say that the trip to Galvanize was worth it.

That, and spears of fresh fruit.

You just can’t argue with fresh fruit spears, where the fruit is cut into various geometrical shapes. It just plucks the right strings of geeky hearts.

Resist Bad Data, Part 2: How to Filter Incomplete XHTML Entities and Encodings

So, as stated in the piece before, dealing with poorly formed XML is a necessary (though infuriating) part of the day for some people, including me. In that article, I described which regular expression to use (with Perl) in order to cleanse your files of such garbage. Which is fine, especially when you’re scripting…but these days, I’m fairly sure that the clear majority of systems are less about batch jobs and more are programming-oriented (distributed architectures, microservices, etc.). So, obviously, it’d be nice to have a solution that could be part of a platform with a more robust programming language.

So, how do we do it in Java? Easy enough. We can just make use of the same regular expression mentioned last time, since Java has a fairly straightforward approach that mimics other languages (like Perl):

—————-

java.nio.file.Path wiki_path = java.nio.file.Paths.get("C:/onix_test_data/test_files", "test_file_ONIX.xml");

java.nio.charset.Charset charset = java.nio.charset.Charset.forName("ISO-8859-1");

try {

StringBuilder FileContents = new StringBuilder();

List lines = java.nio.file.Files.readAllLines(wiki_path, charset);

for (String line : lines) {
     FileContents.append(line + "\n");
}

String sAllFileContents = FileContents.toString();

String sUpdatedFileContents = sAllFileContents.replaceAll("(&#?x?[A-Za-z0-9]+;)|&#\\d*", "$1");

try(java.io.PrintWriter out = new java.io.PrintWriter( "C:/onix_test_data/output/test_file_ONIX.filtered.xml" ) ) {
     out.println( sUpdatedFileContents );
}
} catch (IOException e) {
     System.out.println(e);
}

—————-

And what about C#? Well, as I’ve pointed out again and again, the .NET platform isn’t exactly a source of inspiration when it comes to handling XML. And, of course, it has to be different when it comes to everything, including regular expressions. So, as my personal recommendation, I would make use of regular expressions for pattern matching and make use of the callbacks for actual filtering:

—————-

var sAlteredOutput = Regex.Replace(sOnixTestXml, @"&#?x?[A-Za-z0-9]*;?", FilterMatcher, RegexOptions.Singleline);

...

static public string FilterMatcher(Match m)
{

string sResult = "";

try
{
    if (m.Value == "&#")
        sResult = "";
    else if (m.Value.StartsWith("&#") && (m.Value.Length > 2))
    {
        int nEncodingVal = 0;
        string sInsideEncoding = m.Value.Substring(2);

        if (m.Value.EndsWith(";"))
            sResult = m.Value;
        else if (!Int32.TryParse(sInsideEncoding, out nEncodingVal))
        {
            int nIdx;
            for (nIdx = 0; nIdx < sInsideEncoding.ToCharArray().Length; ++nIdx)
            {
                char cTemp = sInsideEncoding.ToCharArray()[nIdx];

                if (!(cTemp == 'x') && !Char.IsDigit(cTemp)) break;
            }

            if (nIdx == sInsideEncoding.ToCharArray().Length)
                Result = m.Value;
            else
                sResult = m.Value.Substring(nIdx + 2);
        }
        else
            sResult = "";
    }
    else if (m.Value.StartsWith("&"))
        sResult = m.Value;

}
catch (Exception ex)
{
    sResult = m.Value;
}

return sResult;
}

}

—————-

In fact, I’d rather use this solution with callbacks since I love the idea of having more programmatic control. But don’t tell Microsoft that. 😛

And there you go. Two solutions to help you work around the various forms of ineptitude of data suppliers…of which there seems to be no end in sight.