Sunday, April 16, 2017

Tech Epiphanies: A Spotify Listener Looks Longingly at Google Play Music

Uh-oh, Spotify. Google Play Music has some killer features you may want to copy:

  • Family-friendly mode (aka "Protect Jr's Ears" mode).
  • Easy mixing of streamed and owned tracks.
  • Start Song Radio.

As with most tech epiphanies this one occurred while doing something else (managing offspring videos). YouTube Red ads feature prominently. I suspected the offline features of Red might be helpful (this suspicion has not yet been confirmed).

At the bottom of the Red ad it says "includes Google Play Music subscription" [question to future self: any value in dynamic ad element positioning? deal hunters might be more tempted if tucked away where others might not notice]. The first month is free so I decide to give it a spin.

Family-friendly mode (aka "Protect Jr's Ears" mode)

Is a family friendly mode a big deal? Have a look at this Spotify community post. When tens of thousands of people bother to complain about the lack of a feature they likely represent 10x their number as most won't post at all.

Google Play Music has it. And they didn't punt on user convenience, the system itself finds the edited/radio-friendly version of a song and plays that one instead when non-explicit mode is on.

Start Song Radio

All* the streaming services have a "start song radio" feature. It's an attempt to match the convenience of terrestrial radio (turn it on and music starts playing). They already know they've got the terrestrials beat on quality and selection (which is why DJs are coming back... but I digress).

But they all do it awkwardly - if you start listening to the song and then "start song radio" for that song the song is interrupted and either starts over or another is started. Google Play Music does it properly. It's a little thing but the experience is offputting for audiophiles and civilians alike. Great to see it implemented so cleanly.

Thursday, March 2, 2017

TensorFlow Tutorial on Windows 10

These are the steps I took to complete the TensorFlow tutorial on Windows 10 (build 14393).
  1. Download version 3.5.3 64-bit (not 3.6, current latest) from https://www.python.org/ftp/python/3.5.3/python-3.5.3-amd64.exe
    1. Use the custom install and make sure to specify install for all users.
    2. This should change the path to the Program Files directory instead of %LocalAppData%...
    3. Make sure 'add to PATH' is checked.
  2. Open an elevated command prompt and run the following commands (native python method):
    1. pip install -U setuptools
    2. pip install -U wheel
    3. pip install -U --no-cache-dir  https://pypi.python.org/packages/ce/2c/6a1cf90746879c2d05df04efc86a8b1edd79d7b06323a5c8fa63f5520824/tensorflow-1.0.0-cp35-cp35m-win_amd64.whl
Python Interpreter in the Start Menu after Successful Installation
Validating the install produced several errors (e.g., OpKernel(....) for unknown op: BestSplit) during some session runs but still produces the expected 'Hello, TensorFlow!'.

Wednesday, October 26, 2016

Many to Many Entity Framework 6.0 Relationships Without Extra Nesting

So you've got a many to many (m2m) relationship. And you'd like the Entity Framework to return both ends of this relationship using SQL that isn't overly nested/CASE-d/UnionAll-d.

Let's take the canonical m2m relationship of Authors to Books. An Author can write many Books and a Book can be written by many Authors.

An Author

Many Books
Depending on how well the Entity Framework can figure out the multiplicity of each end you may end up with SQL that's very inefficient.

A human would query this with LEFT OUTER JOINs.

But given the following C# code (LINQ to Entities, query syntax):


from a in AUTHORS
select new { a, a.BOOKS }

The generated SQL has too much nesting and aliasing:

SELECT 
"Project1"."C1" AS "C1", 
"Project1"."AUTHOR_ID" AS "AUTHOR_ID", 
"Project1"."FNAME" AS "FNAME", 
"Project1"."LNAME" AS "LNAME", 
"Project1"."C2" AS "C2", 
"Project1"."BOOK_ID" AS "BOOK_ID", 
"Project1"."NAME" AS "NAME", 
"Project1"."DESCRIPTION" AS "DESCRIPTION"
FROM ( SELECT 
 "Extent1"."AUTHOR_ID" AS "AUTHOR_ID", 
 "Extent1"."FNAME" AS "FNAME", 
 "Extent1"."LNAME" AS "LNAME", 
 1 AS "C1", 
 "Join1"."BOOK_ID1" AS "BOOK_ID", 
 "Join1"."NAME" AS "NAME", 
 "Join1"."DESCRIPTION" AS "DESCRIPTION", 
 CASE WHEN ("Join1"."BOOK_ID2" IS NULL) THEN NULL ELSE 1 END AS "C2"
 FROM  "SCOTT"."AUTHORS" "Extent1"
 LEFT OUTER JOIN  (SELECT "Extent2"."BOOK_ID" AS "BOOK_ID2", "Extent2"."AUTHOR_ID" AS "AUTHOR_ID", "Extent3"."BOOK_ID" AS "BOOK_ID1", "Extent3"."NAME" AS "NAME", "Extent3"."DESCRIPTION" AS "DESCRIPTION"
  FROM  "SCOTT"."AUTHOR_BOOKS" "Extent2"
  INNER JOIN "SCOTT"."BOOKS" "Extent3" ON "Extent3"."BOOK_ID" = "Extent2"."BOOK_ID" ) "Join1" ON "Extent1"."AUTHOR_ID" = "Join1"."AUTHOR_ID"
)  "Project1"
ORDER BY "Project1"."AUTHOR_ID" ASC, "Project1"."C2" ASC

This holds true even if eager loading (.Include("BOOKS")) is used. It would be nice to use a LEFT OUTER JOIN here without the extra nesting.

Modified C# to eliminate nesting:

from a in AUTHORS
from b in a.BOOKS.DefaultIfEmpty()
select new { a, b }

Eliminates the extra level of nesting in the resulting SQL:

SELECT 
1 AS "C1", 
"Extent1"."AUTHOR_ID" AS "AUTHOR_ID", 
"Extent1"."FNAME" AS "FNAME", 
"Extent1"."LNAME" AS "LNAME", 
"Join1"."BOOK_ID1" AS "BOOK_ID", 
"Join1"."NAME" AS "NAME", 
"Join1"."DESCRIPTION" AS "DESCRIPTION"
FROM  "SCOTT"."AUTHORS" "Extent1"
LEFT OUTER JOIN  (SELECT "Extent2"."BOOK_ID" AS "BOOK_ID2", "Extent2"."AUTHOR_ID" AS "AUTHOR_ID", "Extent3"."BOOK_ID" AS "BOOK_ID1", "Extent3"."NAME" AS "NAME", "Extent3"."DESCRIPTION" AS "DESCRIPTION"
 FROM  "SCOTT"."AUTHOR_BOOKS" "Extent2"
 INNER JOIN "SCOTT"."BOOKS" "Extent3" ON "Extent3"."BOOK_ID" = "Extent2"."BOOK_ID" ) "Join1" ON "Extent1"."AUTHOR_ID" = "Join1"."AUTHOR_ID"

This example is based on using Entity Framework 6.0 with Oracle's ODP.NET Managed Entity Framework driver. That driver, in turn, depends on the ODP.NET Managed Framework driver (strangely enough, at least at first glance, these are not the same thing).

Wednesday, September 14, 2016

Parser-Friendly Angular 1.3.* Formatting

Angular 1.3.* controllers were initially formatted as follows:

1
2
3
angular.module('path.to.moduleName').controller(['$scope', function($scope) {
  //...
}];

This style has its advantages, particularly while learning angular, in that it reinforces the various concepts (modules, app, dependencies) that are important in the design of the framework.

But unfortunately this style tends to stymie many JavaScript parsers. One widely used library for parsing languages is called Ctags. Many editors rely on this library for parsing. And given a sufficiently long controller using the aforementioned format will easily break the parser.

With a small change in formating we can get the benefit of parsers (e.g., Jump To Definition) while working in JavaScript. With such a free-form language I'll take all of the tool help I can get.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
var myCtrlFunc = function() {
    // ... code goes here
    function fun1() {
        //...
    };
    
    $scope.fun2 = function() {
        //...
    };
};

angular.module('path.to.module').controller('myCtrl', [myCtrlFunc]);

Wednesday, September 7, 2016

What Is Windows Up To Right Now?

Years ago one relied on the hard drive light for reassurance during some particularly unresponsive episode on the PC. The reassuring hum of those spindles spinning formed a kind of non-deterministic progress indicator.

Spin drives gave way to Solid State Drives (SSDs) which afford no reassuring hum. For a while the activity light remained but nowadays it seems that even that has entered the dustbin of PC history. Sort of loses its usefulness if it's always on...

So, like pretty much every commercial concern, I turned to software. Windows has shipped with Resource Monitor (resmon) for several versions. It augments Task Manager's Performance tab (so much so that a button to open it was added to that tab in windows 7).

For answering the general question "What is Windows doing right now?" I maximize the Overview tab on the right monitor. So far the Disk activity grid seems to yield the greatest insight into what's going on at any given instant.

For example, just watching the Disk activity grid while loading a web app revealed excessive logging settings leftover from previous debugging efforts. You can directly see iisexpress.exe writing to the trace logs.

Here are a few tips for getting the most out of resmon's Overview tab:

  1. Increase the height of the Disk grid. It should be, at least, double the height of grids.
  2. Widen the Disk grid's image and file columns. The file column should take up most of the space to account for very long pathnames.
  3. Sort the Disk grid's by Total (B/sec) column in descending order. This brings the files with the most disk activity to the top.
  4. Sort the CPU grid by Average CPU descending.
  5. Sort the Network grid by Total (B/sec) descending.
  6. Sort the Memory grid by Working Set (KB)

Wednesday, August 31, 2016

Single Page Applications And The Impending Concurrency Explosion

Ok explosion might be a little dramatic but bear with me for a moment.

Going from a Multi Page Application (MPA, did I just coin a retronym?) to a Single Page Application (SPA) is like going from a single synchronization lock (coarse-grained locking) to multiple synchronization locks (fine-grained locking).

Anyone who has made that journey knows that it is fraught with peril. Even for a solo developer it can be difficult to implement without error. For a team of developers, with varying level of application domain familiarity and developer expertise, it is far more daunting.


How is going from an MPA to an SPA like this concurrency problem?

With an MPA the full page submit is analogous to acquiring a single lock. The state of the app is updated in one fell swoop.

With an SPA the same amount of state will be distributed across separate async requests that fire, largely, in response to user interaction. Since the user drives the requests any ordering requirements must be enforced by the developer(s).

Each of these requests is like acquiring one of many locks (e.g., read-only consumers only need the read lock, which is shareable, whereas read-write consumers need both the read and write lock (not shareable)). On the user interface, independently queryable data can update different UI widgets in parallel*. This increases the perceived interactivity of the web app but comes at the cost of increased complexity.

It can be difficult to coordinate multiple lock scenarios even when all of the developers working on the project are co-located. Part of this difficulty stems from the difficulty modeling concurrency - there are many notations available and none used by everyone.

Throw in developers distributed geographically, varying levels of expertise, even varying languages and the coordination required to prevent lock-related errors becomes quite substantial.

The theory of it is sound but in practice there are often subtle data dependencies that don't reveal themselves when all data is fetched and the page is rendered in one fell swoop. That is, the dependencies don't show up until teasing apart the data needed for the various dynamic portions of the page. In other words, the data access patterns of an SPA are going to be different than the data access patterns of an MPA.

Wednesday, August 24, 2016

Angular And Filtering Exactly

Angular ships with a set of filters for common output transformations (e.g., formatting a date, formatting currency, etc...).

The filter provider also exposes the 'filter' filter. This filter provides a way to query an array of objects based on property values. It's not as sophisticated as LINQ but it's better than nothing.

By default, the filter 'filter' does a substring match instead of an exact match.

For example, the following code snippet:


$filter('filter')([{name: 'one', male: false}, {name: 'two', male: 'trueisms'}, {name: 'three', male: true}], {male: true})

returns a list with 2 objects (male = 'trueisms' and male = true) instead of 1 (which is what I expected). This is because the default filter will return every object with a male property for which true is a substring.

To get exact matching behavior (and thereby return 1 object in the aforementioned example) pass true as the 3rd argument to the $filter('filter')(...) call.