Here are the slides:

And here’s the video recording:

Here are the slides:

Let’s say you have a Kubernetes cluster and you added nodes containing GPUs to the cluster. Since Kubernetes supports scheduling pods that request GPUs, your data scientists can run ML workloads that need GPUs on your kubernetes cluster. Great!

But you have a problem, sometimes your data scientists come to you and say that their pods requesting GPUs are not getting scheduled. You debug and find out that some other pods (that didn’t request any GPUs) are scheduled on the nodes containing GPUs. So, even though all the GPUs are unused, there’s not enough CPU or memory for the pods requesting GPUs and so they are not getting scheduled. You somehow evict the non-GPU requesting pods from the nodes containing GPUs and pacify your data scientists.

You have another problem, sometimes there are not enough workloads requiring GPUs running on the cluster. Since GPU nodes are the most expensive nodes in your cluster, you want the cluster autoscaler to aggressively downscale these nodes in such a scenario. But that doesn’t seem to happen. You debug and find out the reason for this is the same, some non-GPU requesting pods are scheduled on these nodes. You want a permanent automated solution to this problem.

The solution

Taints is the Kubernetes concept to create dedicated nodes. Taints allow a node to repel a set of pods. Taints work together with tolerations to ensure that pods are not scheduled onto inappropriate nodes. Taints are applied to a node, and mark that the node should not accept any pods that do not tolerate those taints. Tolerations are applied to pods, and allow the pods to schedule onto nodes with matching taints.

So, you decide to taint the nodes containing GPUs and ask your data scientists to modify their GPU pod specs to tolerate that taint. All good? Maybe not. Asking your data scientists to modify their pod specs doesn’t feel like very user friendly. You fear the inevitable situations where someone forgets to apply these tolerations or uses an off-the-shelf manifest that doesn’t have these tolerations.

ExtendedResourceToleration is a new admission controller added in Kubernetes 1.9 that is designed to solve this exact problem.

If you, as a cluster operator or a cloud provider, want to create dedicated node pools, you are expected to taint the nodes containing extended resources (like NVIDIA GPUs) with a key equal to the name of the extended resource (like nvidia.com/gpu) and effect equal to NoSchedule. If you do that, only pods that have a toleration for such a taint can be scheduled on those nodes. To avoid asking your users to modify their pod specs, you can enable the ExtendedResourceToleration admission controller. Then, if your users create a pod that requests extended resources (like nvidia.com/gpu), the admission controller will automatically add a toleration with key equal to the name of the extended resouce (like nvidia.com/gpu), operator Exists and effect NoSchedule to the pod. Because this is happening automatically, this would be invisbile to your users. Their existing pod specs will keep on working. Pods requesting GPUs would not get blocked by non-GPU requesting pods from getting scheduled on GPU nodes. When there are not enough GPU workloads, the cluster autoscaler will downscale GPU nodes (because nothing would be running on them). Everyone will be happy!

The first S3-backed Hadoop filesystem was introduced in Hadoop 0.10.0 (HADOOP-574). It was called the S3 block fileystem and it was assigned the URI scheme s3://. In this implementation, files are stored as blocks, just like they are in HDFS. The files stored by this filesystem are not interoperable with other S3 tools - what this means is that if you go to the AWS console and try to look for files written by this filesystem, you won’t find them - instead you would find files named something like block_-1212312341234512345 etc. Similarly, this filesystem won’t be able to read pre-existing files on S3 since it assumes a block-based format.

To overcome these limitations, another S3-backed filesystem was introduced in Hadoop 0.18.0 (HADOOP-930). It was called the S3 native filesystem and it was assigned the URI scheme s3n://. This filesystem lets you access files on S3 that were written with other tools. Conversely, other tools can access files written using Hadoop. When this filesystem was introduced, S3 had a filesize limit of 5GB and hence this filesystem could only operate with files less than 5GB. In late 2010, Amazon S3 introduced the Multipart Upload API and raised the file size limit from 5GB to 5TB. A lot of Hadoop distributions and HaaS providers added support for this quite early and it landed in Apache Hadoop in version 2.4.0 (HADOOP-9454)

Using the S3 block file system is no longer recommended. Various Hadoop-as-a-service providers like Qubole and Amazon EMR go as far as mapping both the s3:// and the s3n:// URIs to the S3 native filesystem to ensure this.

$ mkdir log-test
$ cd log-test

Create a test java program com/mindprince/MyLogTest.java.

package com.mindprince;

import org.apache.log4j.Logger;

public class MyLogTest {
  static Logger LOG = Logger.getLogger(MyLogTest.class);

  public static void main(String[] args) {
    LOG.debug("Enter main method.");
    FooBarBaz.foo();
    FooBarBaz.bar();
    FooBarBaz.baz();
    LOG.debug("Exit main method.");
  }
}

class FooBarBaz {
  static Logger LOG = Logger.getLogger(FooBarBaz.class);

  public static void foo() {
    LOG.debug("Enter foo method.");
    LOG.info("This is information from foo.");
    LOG.debug("Exit foo method.");
  }

  public static void bar() {
    LOG.debug("Enter bar method.");
    LOG.warn("This is a warning from bar.");
    LOG.debug("Exit bar method.");
  }

  public static void baz() {
    LOG.debug("Enter baz method.");
    LOG.error("This error occurred in baz.");
    LOG.debug("Exit baz method.");
  }
}

Compile the program.

$ export CLASSPATH=$CLASSPATH:.:/path/to/log4j-1.2.17.jar
$ javac com/mindprince/MyLogTest.java

Create a log4j.properties file in the current working directory (log-test/). Log4j will automatically pick this up as this directory is in the classpath.

# Root logger option
log4j.rootLogger=INFO, stdout

# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} [%t] %-5p %c %x - %m%n

$ java com.mindprince.MyLogTest
2013-12-12 14:41:52 [main] INFO  com.mindprince.FooBarBaz  - This is information from foo.
2013-12-12 14:41:52 [main] WARN  com.mindprince.FooBarBaz  - This is a warning from bar.
2013-12-12 14:41:52 [main] ERROR com.mindprince.FooBarBaz  - This error occurred in baz.

Sometimes it is useful to see log4j’s internal debug information.

$ java -Dlog4j.debug=true com.mindprince.MyLogTest
log4j: Trying to find [log4j.xml] using context classloader sun.misc.Launcher$AppClassLoader@1372a1a.
log4j: Trying to find [log4j.xml] using sun.misc.Launcher$AppClassLoader@1372a1a class loader.
log4j: Trying to find [log4j.xml] using ClassLoader.getSystemResource().
log4j: Trying to find [log4j.properties] using context classloader sun.misc.Launcher$AppClassLoader@1372a1a.
log4j: Using URL [file:/home/rohita/log-test/log4j.properties] for automatic log4j configuration.
log4j: Reading configuration from URL file:/home/rohita/log-test/log4j.properties
log4j: Parsing for [root] with value=[INFO, stdout].
log4j: Level token is [INFO].
log4j: Category root set to INFO
log4j: Parsing appender named "stdout".
log4j: Parsing layout options for "stdout".
log4j: Setting property [conversionPattern] to [%d{yyyy-MM-dd HH:mm:ss} [%t] %-5p %c %x - %m%n].
log4j: End of parsing for "stdout".
log4j: Parsed "stdout" options.
log4j: Finished configuring.
2013-12-12 14:42:50 [main] INFO  com.mindprince.FooBarBaz  - This is information from foo.
2013-12-12 14:42:50 [main] WARN  com.mindprince.FooBarBaz  - This is a warning from bar.
2013-12-12 14:42:50 [main] ERROR com.mindprince.FooBarBaz  - This error occurred in baz.

To log messages to a file instead of the console use the following properties:

# Root logger option
log4j.rootLogger=DEBUG, file

# Direct log messages to a log file
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=/path/to/logging.log
log4j.appender.file.MaxFileSize=1MB
# Keep one backup file
log4j.appender.file.MaxBackupIndex=1
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} [%t] %-5p %c %x - %m%n

You can manually specify the location of your log4j properties file using -Dlog4j.configuration

$ java -Dlog4j.configuration=file:/path/to/log4j-tofile.properties com.mindprince.MyLogTest

$ cat /path/to/logging.log
2013-12-12 14:44:07 [main] DEBUG com.mindprince.MyLogTest  - Enter main method.
2013-12-12 14:44:07 [main] DEBUG com.mindprince.FooBarBaz  - Enter foo method.
2013-12-12 14:44:07 [main] INFO  com.mindprince.FooBarBaz  - This is information from foo.
2013-12-12 14:44:07 [main] DEBUG com.mindprince.FooBarBaz  - Exit foo method.
2013-12-12 14:44:07 [main] DEBUG com.mindprince.FooBarBaz  - Enter bar method.
2013-12-12 14:44:07 [main] WARN  com.mindprince.FooBarBaz  - This is a warning from bar.
2013-12-12 14:44:07 [main] DEBUG com.mindprince.FooBarBaz  - Exit bar method.
2013-12-12 14:44:07 [main] DEBUG com.mindprince.FooBarBaz  - Enter baz method.
2013-12-12 14:44:07 [main] ERROR com.mindprince.FooBarBaz  - This error occurred in baz.
2013-12-12 14:44:07 [main] DEBUG com.mindprince.FooBarBaz  - Exit baz method.
2013-12-12 14:44:07 [main] DEBUG com.mindprince.MyLogTest  - Exit main method.

Sometimes it is useful to change the logging behaviour for particular classes:

# Root logger option
log4j.rootLogger=DEBUG, stdout

# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} [%t] %-5p %c %x - %m%n

# Print only messages of level WARN or above for FooBarBaz
log4j.logger.com.mindprince.FooBarBaz=WARN

$ java -Dlog4j.configuration=file:/path/to/log4j-selective.properties com.mindprince.MyLogTest
2013-12-12 14:51:36 [main] DEBUG com.mindprince.MyLogTest  - Enter main method.
2013-12-12 14:51:36 [main] WARN  com.mindprince.FooBarBaz  - This is a warning from bar.
2013-12-12 14:51:36 [main] ERROR com.mindprince.FooBarBaz  - This error occurred in baz.
2013-12-12 14:51:36 [main] DEBUG com.mindprince.MyLogTest  - Exit main method.

I usually log everything to a file but only log important messages to the console:

# Root logger option
log4j.rootLogger=DEBUG, stdout, file

# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} [%t] %-5p %c %x - %m%n
# Print only messages of level WARN or above on the console.
log4j.appender.stdout.Threshold=WARN

# Direct log messages to a log file
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=/path/to/messages.log
log4j.appender.file.MaxFileSize=1MB
log4j.appender.file.MaxBackupIndex=1
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} [%t] %-5p %c %x - %m%n

$ java -Dlog4j.configuration=file:/path/to/log4j-both.properties com.mindprince.MyLogTest
2013-12-12 15:10:31 [main] WARN  com.mindprince.FooBarBaz  - This is a warning from bar.
2013-12-12 15:10:31 [main] ERROR com.mindprince.FooBarBaz  - This error occurred in baz.

$ cat /path/to/messages.log
2013-12-12 15:10:31 [main] DEBUG com.mindprince.MyLogTest  - Enter main method.
2013-12-12 15:10:31 [main] DEBUG com.mindprince.FooBarBaz  - Enter foo method.
2013-12-12 15:10:31 [main] INFO  com.mindprince.FooBarBaz  - This is information from foo.
2013-12-12 15:10:31 [main] DEBUG com.mindprince.FooBarBaz  - Exit foo method.
2013-12-12 15:10:31 [main] DEBUG com.mindprince.FooBarBaz  - Enter bar method.
2013-12-12 15:10:31 [main] WARN  com.mindprince.FooBarBaz  - This is a warning from bar.
2013-12-12 15:10:31 [main] DEBUG com.mindprince.FooBarBaz  - Exit bar method.
2013-12-12 15:10:31 [main] DEBUG com.mindprince.FooBarBaz  - Enter baz method.
2013-12-12 15:10:31 [main] ERROR com.mindprince.FooBarBaz  - This error occurred in baz.
2013-12-12 15:10:31 [main] DEBUG com.mindprince.FooBarBaz  - Exit baz method.
2013-12-12 15:10:31 [main] DEBUG com.mindprince.MyLogTest  - Exit main method.

1. Open the terminal.

2. In the termial, go to the folder which contains the files you want to transfer.

3. Enter python -m SimpleHTTPServer 8000.

4. If the firewall prompts: Do you want the application “python” to accept incoming network connections? Click Allow.

5. Find out the IP address of your Mac using ifconfig.

6. Go to http://<your-mac's-ip-address>:8000 on your iPad.

7. Done!

This method is very general. You can use it to transfer files from your Mac or PC to your iPad, iPhone or Android device. The only requirements are that they should be on the same network, the Mac/PC should have python installed and you should have permission to poke holes in your firewall.

A lot of advertisers are part of self-regulatory programs and let you opt-out from receiving interest-based advertising. The following tools let you find out which advertisers have enabled interest-based ads for your browser and opt-out from them. These tools use opt-out cookies to store your preferences in your browser. (Note that if you are running advertising blocking plugins, these tools may not work correctly.)

Digital Advertising Alliance (DAA) opt-out tool
Network Advertising Initaitive (NAI) opt-out tool
EDAA Your Online Choices opt-out tool

Since the opt-out preferences are stored as cookies, deleting cookies will remove your opt-out preferences. You can use the following extensions to preserve the opt-out preferences you have set.

Protect My Choices browser extension
Keep My Opt-Outs browser extension

Analytics

Various analytics services provide ways to opt-out from getting tracked by them. The following are the opt-out pages of the major analytics providers:

Google Analytics Opt-out Browser Add-on [1]
Yahoo Web Analytics Opt-out
KISSmetrics Opt-out [2]
MixPanel Opt-out
Converto Opt-out
Webtrends Opt-out

All-in-one Browser Extensions

The following extensions block tracking requests from analytics services, advertising networks and social networking sites. They are a more drastic measure for blocking tracking as they can break the functionality of some web pages. It is recommended that you use only one of these as they interfere with each other’s working.

Disconnect
Ghostery
Abine DoNotTrackMe

[1] This blog uses Google Analytics. I know.
[2] I like how the the opt-out page of KISSmetrics links to the opt-out pages of its competitors.

Command-line arguments in Python

import sys
# sys.argv is a list of strings.
# argv[0] is the script name
for arg in sys.argv:
    # do something with the argument
    print arg

$ python cmd_line_args.py Position1 Position2 Position3
cmd_line_args.py
Position1
Position2
Position3
$ python cmd_line_args.py "Position1 Position2 Position3"
cmd_line_args.py
Position1 Position2 Position3

import sys
# sys.argv is a list of strings.
# argv[0] is the script name
for arg in sys.argv[1:]:
    # if you are sure that the arguments are integers
    arg_int = int(arg)
    # do something with the argument
    print arg_int

$ python cmd_line_args.py 2013 6 30
2013
6
30
$ python cmd_line_args.py Position1 Position2 Position3
Traceback (most recent call last):
  File "cmd_line_args.py", line 6, in <module>
    arg_int = int(arg)
ValueError: invalid literal for int() with base 10: 'Position1'

Reference: Python Tutorial.

For more advanced needs, you can try the argparse module.

Command-line arguments in Java

public class CommandLineArguments {
    public static void main(String[] args) {
        // args is an array of Strings
        // You can access various positional arguments using
        // args[0], args[1], etc. Or you can iterate over them.
        for (String argument : args) {
            // do something with the argument
            System.out.println(argument);
        }
    }
}

$ javac CommandLineArguments.java 
$ java CommandLineArguments Position1 Position2 Position3
Position1
Position2
Position3
$ java CommandLineArguments "Position1 Position2 Position3"
Position1 Position2 Position3

public class CommandLineArgumentsInt {
    public static void main(String[] args) {
        // args is an array of Strings
        // You can access various positional arguments using
        // args[0], args[1], etc. Or you can iterate over them.
        for (String argument : args) {
            // If you are sure that the arguments are integers
            int arg = Integer.parseInt(argument);
            // do something with the argument
            System.out.println(arg);
        }
    }
}

$ javac CommandLineArgumentsInt.java 
$ java CommandLineArgumentsInt 2013 6 30
2013
6
30
$ java CommandLineArgumentsInt Position1 Position2 Position3
Exception in thread "main" java.lang.NumberFormatException: For input string: "Position1"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
	at java.lang.Integer.parseInt(Integer.java:449)
	at java.lang.Integer.parseInt(Integer.java:499)
	at CommandLineArgumentsInt.main(CommandLineArgumentsInt.java:8)

Reference: The Java Tutorial.

For more advanced needs, see this Stack Overflow question.

Command-line arguments in C

#include <stdio.h>

int main(int argc, char* argv[]) {
    // argc refers to the number of arguments passed.
    // argv is an array of char pointers which contain the passed arguments.
    // argv[0] is the path and name of the program itself
    for (int i = 0; i < argc; i++) {
        // do something whith the argument
        printf("%s\n", argv[i]);
    }
}

$ cc cmd_line_args.c 
$ ./a.out Position1 Position2 Position3
./a.out
Position1
Position2
Position3
$ ./a.out "Position1 Position2 Position3"
./a.out
Position1 Position2 Position3

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char* argv[]) {
    // argc refers to the number of arguments passed.
    // argv is an array of char pointers which contain the passed arguments.
    // argv[0] is the path and name of the program itself
    for (int i = 1; i < argc; i++) {
        // If you are sure that the arguments are integers
        int argument = atoi(argv[i]);

        // do something whith the argument
        printf("%d\n", argument);
    }
}

$ cc cmd_line_args.c 
$ ./a.out 2013 6 30
2013
6
30
$ ./a.out Position1 Position2 Position3
0
0
0

For more advanced needs, use getopt.

Reading a file in Python

Reading a file at once

try:
    with open("/path/to/file/filename.txt", "r") as f:
        text = f.read()
        # text is a string which contains all the text in the file.
        # this may cause problems with very big files.

        # do something with text
        print text
except IOError as e:
    print "IOError: %s: %s" % (e.filename, e.strerror)

Reading a file line by line

try:
    with open("/path/to/file/filename.txt", "r") as f:
        for line in f:
            # line is a string
            # a newline character (\n) is present at the end of the string
            # and is only omitted on the last line of the file if the
            # file doesn't end in a newline.

            # do something with line
            print line
except IOError as e:
    print "IOError: %s: %s" % (e.filename, e.strerror)

Reading a file word by word

try:
    with open("/path/to/file/filename.txt", "r") as f:
        for line in f:
            for word in line.split():
                # do something with word
                print word
except IOError as e:
    print "IOError: %s: %s" % (e.filename, e.strerror)

Reading a file character by character

try:
    with open("/path/to/file/filename.txt", "r") as f:
        while True:
            character = f.read(1)
            if character:
                # do something with character
                print character
            else:
                # end-of-file reached
                break
except IOError as e:
    print "IOError: %s: %s" % (e.filename, e.strerror)

Writing a file in Python

with open("/path/to/file/filename.txt", "w") as f:
    # write to the file using f.write()
    f.write('Whatever you want to write.\n')
    f.write('Really, anything.\n')

Reading a file in Java

Reading a file at once

This question on Stack Overflow shows various ways to do this.

Reading a file line by line

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.FileNotFoundException;
import java.util.Scanner;

public class FileLL {
    public static void main(String[] args) {
        Scanner s = null;
        try {
            s = new Scanner(new BufferedReader(new FileReader("/path/to/file/filename.txt")));
            while (s.hasNext()) {
                String line = s.nextLine();
                // line does not contain a newline (\n) at the end.

                // do something with line
                System.out.print(line);
            }
        } catch (FileNotFoundException e) {
            System.err.println("Can't open input file!");
            System.err.println(e.getMessage());
        } finally {
            if (s != null) {
                s.close();
            }
        }
    }
}

Reading a file word by word

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.FileNotFoundException;
import java.util.Scanner;

public class FileWW {
    public static void main(String[] args) {
        Scanner s = null;
        try {
            s = new Scanner(new BufferedReader(new FileReader("/path/to/file/filename.txt")));
            while (s.hasNext()) {
                String word = s.next();

                // do something with the word
                System.out.println(word);
            }
        } catch (FileNotFoundException e) {
            System.err.println("Can't open input file!");
            System.err.println(e.getMessage());
        } finally {
            if (s != null) {
                s.close();
            }
        }
    }
}

Reading a file character by character

import java.io.FileReader;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.IOException;

public class FileCC {
    public static void main(String[] args) throws IOException {
        BufferedReader br = null;
        try {
            br = new BufferedReader(new FileReader("/path/to/file/filename.txt"));
            int ch;
            while ((ch = br.read()) != -1) {
                char character = (char) ch;

                // do something with the character.
                System.out.print(character);
            }
        } catch (FileNotFoundException e) {
            System.err.println("Can't open input file!");
            System.err.println(e.getMessage());
        } finally {
            if (br != null) {
                br.close();
            }
        }
    }
}

Writing a file in Java

import java.io.FileWriter;
import java.io.BufferedWriter;
import java.io.IOException;

public class FileW {
    public static void main(String[] args) throws IOException {
        BufferedWriter bw = new BufferedWriter(new FileWriter("/path/to/file/filename.txt"));
        bw.write("Whatever you want to write.\n");
        bw.write("Really, anything.\n");
        bw.close();
    }
}

Reading a file in C

Reading a file at once

I don’t know of a straightforward way to do this. Feel free to chime in the comments if you know one.

Reading a file line by line

#include <stdlib.h>
#include <stdio.h>

int main() {

    FILE *fp = fopen("/path/to/file/filename.txt", "r");
    if (fp == NULL) {
        fprintf(stderr, "Can't open input file!\n");
        exit(1);
    }

    char line[1024];
    // assuming no line in the file is longer than 1024 characters.
    while (fgets(line, sizeof line, fp) != NULL) {
        // line contains a newline (\n) at the end
        // except maybe for the last line of the file
        // if the file doesn't end with a newline

        // do something with line
        printf("%s\n", line);
    }

    fclose(fp);

    return 0;
}

Reading a file word by word

#include <stdlib.h>
#include <stdio.h>

int main() {

    FILE *fp = fopen("/path/to/file/filename.txt", "r");
    if (fp == NULL) {
        fprintf(stderr, "Can't open input file!\n");
        exit(1);
    }

    char word[64];
    // assuming no word in the file is longer than 64 characters.
    while (fscanf(fp, "%64s", word) != EOF) {
        // do something with word
        printf("%s\n", word);
    }

    fclose(fp);

    return 0;
}

Reading a file character by character

#include <stdlib.h>
#include <stdio.h>

int main() {

    FILE *fp = fopen("/path/to/file/filename.txt", "r");
    if (fp == NULL) {
        fprintf(stderr, "Can't open input file!\n");
        exit(1);
    }

    int ch;
    while ((ch = fgetc(fp)) != EOF) {
        char character = (char) ch;
        // do something with character
        printf("%c", character);
    }

    fclose(fp);

    return 0;
}

Writing a file in C

#include <stdlib.h>
#include <stdio.h>

int main() {

    FILE *fp = fopen("/path/to/file/filename.txt", "w");
    if (fp == NULL) {
        fprintf(stderr, "Can't open input file!\n");
        exit(1);
    }
    fprintf(fp, "Whatever you want to write.\n");
    fprintf(fp, "Really, anything.\n");
    fclose(fp);

    return 0;
}

Here’s the summary:

Unrooted Android devices don’t support ad-hoc mode https://code.google.com/p/android/issues/detail?id=82 i.e. they don’t see ad-hoc WiFi networks.

iOS devices support ad-hoc mode i.e. they can see ad-hoc WiFi networks.

Now, if you are on a Mac and you want to share your internet connection, you follow the steps outlined here: http://support.apple.com/kb/PH3853, this creates a WiFi network in Infrastructure mode which is visible to both iOS and Android devices and you can access the internet on these devices using this network.

You can create an ad-hoc network on a Mac by following these steps: http://support.apple.com/kb/PH3771. But this network is not visible to unrooted Android devices and iOS devices were not able to access the internet even after connecting to this network. So, don’t do this.

If you are on a PC (Windows 7 or later), you can use Connectify Hotspot to create a hotspot in Access Point Mode which is visible to both Android and iOS devices and access the internet on these devices using this hotspot. However, if you are on Windows XP or Vista, your Connectify hotspot will operate in ad-hoc mode and will not work with unrooted Android devices [1] [2].

If you don’t want to use Connectify Hotspot, you can create an ad-hoc network by following these steps: http://windows.microsoft.com/en-in/windows7/set-up-a-computer-to-computer-ad-hoc-network. This network is not visible to unrooted Android devices but is visible to iOS devices and they can access the internet using this network.

If you are on a PC running Ubuntu, you can create an ad-hoc network by going to the Network menu and then clicking Create New Wireless Network. This network is not visible to unrooted Android devices but is visible to iOS devices and they can access the internet using this network.

I don’t know of a way to create a WiFi network on an Ubuntu machine which is visible to unrooted Android devices.

0   1   0   0   9                1   0   0                0   0   0
0   0   3   0   0       \ /      0   0   0      -----     0  21   0
0   0   0   2   0        *       0   7   0      -----     0   0   4
0   0   0   0   0       / \      0   0   2                0   0   0
                                 0   0   0

We can represent a sparse matrix in a relational database as a table matrix_name(row_num, col_num, value). Each non-zero cell in the matrix is represnted as a record (i, j, value) in the table.

Here’s how to do the multiplication. First, create the tables:

CREATE TABLE a (
row_num INT,
col_num INT,
value INT,
PRIMARY KEY(row_num, col_num)
); 

CREATE TABLE b (
row_num INT,
col_num INT,
value INT,
PRIMARY KEY(row_num, col_num)
); 

Next, insert values into the tables:

INSERT INTO a VALUES
(1, 2, 1),
(1, 5, 9),
(2, 3, 3),
(3, 4, 2);

INSERT INTO b VALUES
(1, 1, 1),
(3, 2, 7),
(4, 3, 2);

Finally, perform the multiplication using the following query:

SELECT a.row_num, b.col_num, SUM(a.value*b.value)
FROM a, b
WHERE a.col_num = b.row_num
GROUP BY a.row_num, b.col_num;

2|2|21
3|3|4

To see how this query works, remember the formula for cell (i,j) of the product. It is the sum of a(i,k)*b(k,j) for all k. The JOIN condition a.col_num = b.row_num makes sure that both a.value and b.value has the same k. The GROUP BY clause makes sure that we sum over all k’s.

So, the first step is to download an OAuth library. I am going to use simplegeo/python-oauth2. You can download it from pypi and install it by executing sudo ./setup.py install.

Here is a script which uses the API to search twitter from the terminal.

To use this script, you first need to create an application at https://dev.twitter.com/apps

You will get your consumer_key and consumer_secret from there. You will also need to create an access token for your account. After doing that you will get your access_token_key and access_token_secret.

Fill these four values in twitter_search.py and you are good to go.

$ python twitter_search.py SEARCH TERM(S)

$ git branch
* source

To understand what was going on, I dug up the docs.

Initially when you clone the octopress repo from imathis/octopress, the local branch is called master and the remote imathis/octopress is called origin.

The magic happens when you do rake setup_github_pages.

It renames the remote pointing to imathis/octopress from origin to octopress; adds your Github Pages repository as the default origin remote and renames the master branch to source. It then creates a new directory _deploy/ and initializes a new git repository there. Finally, it adds your Github Pages repository as the default origin remote for this new repository also.

Now, when you do rake deploy, the master branch of the git repo in _deploy/ is pushed to the master branch of your Github Pages repo. And when you push the source repo to github, it becomes the source branch of your Github Pages repo.

Here are some more details:

$ cd octopress/  # THE_DIRECTORY_WHERE_YOU_CLONED_OCTOPRESS
$ git remote show origin
* remote origin
  Fetch URL: git@github.com:mindprince/mindprince.github.io.git
  Push  URL: git@github.com:mindprince/mindprince.github.io.git
  HEAD branch: master
  Remote branches:
    master new (next fetch will store in remotes/origin)
    source tracked
  Local ref configured for 'git push':
    source pushes to source (up to date)

$ cd octopress/_deploy/
$ git remote show origin
* remote origin
  Fetch URL: git@github.com:mindprince/mindprince.github.io.git
  Push  URL: git@github.com:mindprince/mindprince.github.io.git
  HEAD branch: master
  Remote branches:
    master tracked
    source new (next fetch will store in remotes/origin)
  Local ref configured for 'git push':
    master pushes to master (up to date)

To summarize, the two branches source and master on your Github Pages repo correspond to two separate git repositories: one in your octopress/ directory and another in the octopress/_deploy/ directory.

The fact that separate git repositories on your local machine can act as different branches of the same repository on github is interesting. It can probably be misused to host more than five private repositories in Github’s Micro plan.

a is a string encoded in utf-8.

>>> b = unicode(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 9: ordinal not in range(128)

This didn’t work because the default encoding in python in ascii. So, python was not able to decode a assuming ascii encoding.

>>> b = unicode(a, "utf-8")
>>> type(b)
<type 'unicode'>
>>> b
u"He said, \u201cHi, there.\u201d She didn't reply."
>>> print b
He said, “Hi, there.” She didn't reply.

b is not a string. It is a unicode object. I think it has no encoding. You can encode it in different encodings.

>>> c = b.encode("utf-8")
>>> type(c)
<type 'str'>
>>> c
"He said, \xe2\x80\x9cHi, there.\xe2\x80\x9d She didn't reply."
>>> print c
He said, “Hi, there.” She didn't reply.

c is now same as a. It is a string encoded in utf-8. We created it by encoding a unicode object.

>>> d = a.encode("utf-8")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 9: ordinal not in range(128)

a is already encoded in utf-8. What happens here is that python first tries to decode a and then encode a. But decoding a fails because default encoding is assume to be ascii.

>>> e = a.decode("utf-8")
>>> type(e)
<type 'unicode'>
>>> e
u"He said, \u201cHi, there.\u201d She didn't reply."
>>> print e
He said, “Hi, there.” She didn't reply.

Now, e is same as b. It is a unicode object.

>>> f = a.decode("ascii")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 9: ordinal not in range(128)

Just to show what we have been saying earlier.

>>> g = b.encode("ascii")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position 9: ordinal not in range(128)

Note this is an UnicodeEncodeError and not a UnicodeDecodeError. We can’t encode a unicode object which contains characters outside of ascii range to ascii encoding.