Thursday, March 2, 2017

TensorFlow Tutorial on Windows 10

These are the steps I took to complete the TensorFlow tutorial on Windows 10 (build 14393).
  1. Download version 3.5.3 64-bit (not 3.6, current latest) from https://www.python.org/ftp/python/3.5.3/python-3.5.3-amd64.exe
    1. Use the custom install and make sure to specify install for all users.
    2. This should change the path to the Program Files directory instead of %LocalAppData%...
    3. Make sure 'add to PATH' is checked.
  2. Open an elevated command prompt and run the following commands (native python method):
    1. pip install -U setuptools
    2. pip install -U wheel
    3. pip install -U --no-cache-dir  https://pypi.python.org/packages/ce/2c/6a1cf90746879c2d05df04efc86a8b1edd79d7b06323a5c8fa63f5520824/tensorflow-1.0.0-cp35-cp35m-win_amd64.whl
Python Interpreter in the Start Menu after Successful Installation
Validating the install produced several errors (e.g., OpKernel(....) for unknown op: BestSplit) during some session runs but still produces the expected 'Hello, TensorFlow!'.

Wednesday, October 26, 2016

Many to Many Entity Framework 6.0 Relationships Without Extra Nesting

So you've got a many to many (m2m) relationship. And you'd like the Entity Framework to return both ends of this relationship using SQL that isn't overly nested/CASE-d/UnionAll-d.

Let's take the canonical m2m relationship of Authors to Books. An Author can write many Books and a Book can be written by many Authors.

An Author

Many Books
Depending on how well the Entity Framework can figure out the multiplicity of each end you may end up with SQL that's very inefficient.

A human would query this with LEFT OUTER JOINs.

But given the following C# code (LINQ to Entities, query syntax):


from a in AUTHORS
select new { a, a.BOOKS }

The generated SQL has too much nesting and aliasing:

SELECT 
"Project1"."C1" AS "C1", 
"Project1"."AUTHOR_ID" AS "AUTHOR_ID", 
"Project1"."FNAME" AS "FNAME", 
"Project1"."LNAME" AS "LNAME", 
"Project1"."C2" AS "C2", 
"Project1"."BOOK_ID" AS "BOOK_ID", 
"Project1"."NAME" AS "NAME", 
"Project1"."DESCRIPTION" AS "DESCRIPTION"
FROM ( SELECT 
 "Extent1"."AUTHOR_ID" AS "AUTHOR_ID", 
 "Extent1"."FNAME" AS "FNAME", 
 "Extent1"."LNAME" AS "LNAME", 
 1 AS "C1", 
 "Join1"."BOOK_ID1" AS "BOOK_ID", 
 "Join1"."NAME" AS "NAME", 
 "Join1"."DESCRIPTION" AS "DESCRIPTION", 
 CASE WHEN ("Join1"."BOOK_ID2" IS NULL) THEN NULL ELSE 1 END AS "C2"
 FROM  "SCOTT"."AUTHORS" "Extent1"
 LEFT OUTER JOIN  (SELECT "Extent2"."BOOK_ID" AS "BOOK_ID2", "Extent2"."AUTHOR_ID" AS "AUTHOR_ID", "Extent3"."BOOK_ID" AS "BOOK_ID1", "Extent3"."NAME" AS "NAME", "Extent3"."DESCRIPTION" AS "DESCRIPTION"
  FROM  "SCOTT"."AUTHOR_BOOKS" "Extent2"
  INNER JOIN "SCOTT"."BOOKS" "Extent3" ON "Extent3"."BOOK_ID" = "Extent2"."BOOK_ID" ) "Join1" ON "Extent1"."AUTHOR_ID" = "Join1"."AUTHOR_ID"
)  "Project1"
ORDER BY "Project1"."AUTHOR_ID" ASC, "Project1"."C2" ASC

This holds true even if eager loading (.Include("BOOKS")) is used. It would be nice to use a LEFT OUTER JOIN here without the extra nesting.

Modified C# to eliminate nesting:

from a in AUTHORS
from b in a.BOOKS.DefaultIfEmpty()
select new { a, b }

Eliminates the extra level of nesting in the resulting SQL:

SELECT 
1 AS "C1", 
"Extent1"."AUTHOR_ID" AS "AUTHOR_ID", 
"Extent1"."FNAME" AS "FNAME", 
"Extent1"."LNAME" AS "LNAME", 
"Join1"."BOOK_ID1" AS "BOOK_ID", 
"Join1"."NAME" AS "NAME", 
"Join1"."DESCRIPTION" AS "DESCRIPTION"
FROM  "SCOTT"."AUTHORS" "Extent1"
LEFT OUTER JOIN  (SELECT "Extent2"."BOOK_ID" AS "BOOK_ID2", "Extent2"."AUTHOR_ID" AS "AUTHOR_ID", "Extent3"."BOOK_ID" AS "BOOK_ID1", "Extent3"."NAME" AS "NAME", "Extent3"."DESCRIPTION" AS "DESCRIPTION"
 FROM  "SCOTT"."AUTHOR_BOOKS" "Extent2"
 INNER JOIN "SCOTT"."BOOKS" "Extent3" ON "Extent3"."BOOK_ID" = "Extent2"."BOOK_ID" ) "Join1" ON "Extent1"."AUTHOR_ID" = "Join1"."AUTHOR_ID"

This example is based on using Entity Framework 6.0 with Oracle's ODP.NET Managed Entity Framework driver. That driver, in turn, depends on the ODP.NET Managed Framework driver (strangely enough, at least at first glance, these are not the same thing).

Wednesday, September 14, 2016

Parser-Friendly Angular 1.3.* Formatting

Angular 1.3.* controllers were initially formatted as follows:

1
2
3
angular.module('path.to.moduleName').controller(['$scope', function($scope) {
  //...
}];

This style has its advantages, particularly while learning angular, in that it reinforces the various concepts (modules, app, dependencies) that are important in the design of the framework.

But unfortunately this style tends to stymie many JavaScript parsers. One widely used library for parsing languages is called Ctags. Many editors rely on this library for parsing. And given a sufficiently long controller using the aforementioned format will easily break the parser.

With a small change in formating we can get the benefit of parsers (e.g., Jump To Definition) while working in JavaScript. With such a free-form language I'll take all of the tool help I can get.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
var myCtrlFunc = function() {
    // ... code goes here
    function fun1() {
        //...
    };
    
    $scope.fun2 = function() {
        //...
    };
};

angular.module('path.to.module').controller('myCtrl', [myCtrlFunc]);

Wednesday, September 7, 2016

What Is Windows Up To Right Now?

Years ago one relied on the hard drive light for reassurance during some particularly unresponsive episode on the PC. The reassuring hum of those spindles spinning formed a kind of non-deterministic progress indicator.

Spin drives gave way to Solid State Drives (SSDs) which afford no reassuring hum. For a while the activity light remained but nowadays it seems that even that has entered the dustbin of PC history. Sort of loses its usefulness if it's always on...

So, like pretty much every commercial concern, I turned to software. Windows has shipped with Resource Monitor (resmon) for several versions. It augments Task Manager's Performance tab (so much so that a button to open it was added to that tab in windows 7).

For answering the general question "What is Windows doing right now?" I maximize the Overview tab on the right monitor. So far the Disk activity grid seems to yield the greatest insight into what's going on at any given instant.

For example, just watching the Disk activity grid while loading a web app revealed excessive logging settings leftover from previous debugging efforts. You can directly see iisexpress.exe writing to the trace logs.

Here are a few tips for getting the most out of resmon's Overview tab:

  1. Increase the height of the Disk grid. It should be, at least, double the height of grids.
  2. Widen the Disk grid's image and file columns. The file column should take up most of the space to account for very long pathnames.
  3. Sort the Disk grid's by Total (B/sec) column in descending order. This brings the files with the most disk activity to the top.
  4. Sort the CPU grid by Average CPU descending.
  5. Sort the Network grid by Total (B/sec) descending.
  6. Sort the Memory grid by Working Set (KB)

Wednesday, August 31, 2016

Single Page Applications And The Impending Concurrency Explosion

Ok explosion might be a little dramatic but bear with me for a moment.

Going from a Multi Page Application (MPA, did I just coin a retronym?) to a Single Page Application (SPA) is like going from a single synchronization lock (coarse-grained locking) to multiple synchronization locks (fine-grained locking).

Anyone who has made that journey knows that it is fraught with peril. Even for a solo developer it can be difficult to implement without error. For a team of developers, with varying level of application domain familiarity and developer expertise, it is far more daunting.


How is going from an MPA to an SPA like this concurrency problem?

With an MPA the full page submit is analogous to acquiring a single lock. The state of the app is updated in one fell swoop.

With an SPA the same amount of state will be distributed across separate async requests that fire, largely, in response to user interaction. Since the user drives the requests any ordering requirements must be enforced by the developer(s).

Each of these requests is like acquiring one of many locks (e.g., read-only consumers only need the read lock, which is shareable, whereas read-write consumers need both the read and write lock (not shareable)). On the user interface, independently queryable data can update different UI widgets in parallel*. This increases the perceived interactivity of the web app but comes at the cost of increased complexity.

It can be difficult to coordinate multiple lock scenarios even when all of the developers working on the project are co-located. Part of this difficulty stems from the difficulty modeling concurrency - there are many notations available and none used by everyone.

Throw in developers distributed geographically, varying levels of expertise, even varying languages and the coordination required to prevent lock-related errors becomes quite substantial.

The theory of it is sound but in practice there are often subtle data dependencies that don't reveal themselves when all data is fetched and the page is rendered in one fell swoop. That is, the dependencies don't show up until teasing apart the data needed for the various dynamic portions of the page. In other words, the data access patterns of an SPA are going to be different than the data access patterns of an MPA.

Wednesday, August 24, 2016

Angular And Filtering Exactly

Angular ships with a set of filters for common output transformations (e.g., formatting a date, formatting currency, etc...).

The filter provider also exposes the 'filter' filter. This filter provides a way to query an array of objects based on property values. It's not as sophisticated as LINQ but it's better than nothing.

By default, the filter 'filter' does a substring match instead of an exact match.

For example, the following code snippet:


$filter('filter')([{name: 'one', male: false}, {name: 'two', male: 'trueisms'}, {name: 'three', male: true}], {male: true})

returns a list with 2 objects (male = 'trueisms' and male = true) instead of 1 (which is what I expected). This is because the default filter will return every object with a male property for which true is a substring.

To get exact matching behavior (and thereby return 1 object in the aforementioned example) pass true as the 3rd argument to the $filter('filter')(...) call.


Wednesday, August 17, 2016

Chrome Tip - Searching Every/All Source Files At Once

To search through every source file loaded by the currently viewed web page (e.g., the HTML, JavaScript and CSS) open Chrome Dev Tools. From the Sources tab right click topmost node in the tree then choose "Search in all files".

Given that it's accessed via an element in a tree one wonders if it can be used to search through all source files under a given node? A quick test confirms this!