Unicode for Python on Windows

太久沒有用 windows 了,這個問題花了我一點時間,重點在下面的設定:

I/O Encoding

@chcp 65001 > nul
@set PYTHONIOENCODING=utf-8

下面是檢查 stdin/stdout 的 encoding

$ python -c 'import sys; print sys.stdin.encoding, sys.stdout.encoding'

Default Encoding

修改 default encoding 從 ascii -> utf-8

import sys
print sys.getdefaultencoding()
reload(sys)
sys.setdefaultencoding('utf-8')
print sys.getdefaultencoding()

詳細可以參考 Python 2 on Windows: how to read command line arguments containing Unicode code points

Ruby Environment on Mavericks

Prerequisite

  • Homebrew
  • openssl, readline, rbenv, ruby-build
brew install readline openssl rbenv ruby-build rbenv-gem-rehash

Ruby 2.X

You may want to refer to these issues for more details on installing 2.X:

Here I prefer the following:

curl -fsSL https://gist.github.com/EdVanDance/9738787.txt | RUBY_CONFIGURE_OPTS="--with-readline-dir=$(brew --prefix readline) --with-openssl-dir=$(brew --prefix openssl) --with-gcc=clang" rbenv install --patch 2.0.0-p451

Then set as default and install some utils:

rbenv global 2.0.0-p451
rbenv rehash
gem install bundler
rbenv rehash

Finder Hook Points in OS X

Contextual Menu

[TContextMenu +addViewSpecificStuffToMenu:browserViewController:context:] is for handling selected items cached in [TContextMenu +handleContextMenuCommon:nodes:event:view:windowController:addPlugIns:] (10.6 & 10.7) or [TContextMenu +handleContextMenuCommon:nodes:event:view:browserController:addPlugIns:] (10.8)

10.6 & 10.7

  • [TContextMenu +handleContextMenuCommon:nodes:event:view:windowController:addPlugIns:]: cached selected items for later use. Because only at this point we can really know what those items are.
  • [TContextMenu -configureWithNodes:windowController:container:]: at this point, we can know the selected items and directly append additional menu.

10.8

Same as in 10.6 & 10.7 but different names of hook points

  • [TContextMenu +handleContextMenuCommon:nodes:event:view:browserController:addPlugIns:]
  • [TContextMenu -configureWithNodes:browserController:container:]

10.9

In Mavericks, Finder has massive changes. The previous hook points are no longer available. But new hook point makes me easier to do things.

  • [TContextMenu -configureFromMenuNeedsUpdate:clickedView:container:event:selectedNodes:]: at this point, thank god we got selected items finally. I've been wondering why not Apple put this until 10.9.

Icon Overlay

Icon View

  • [IKImageBrowserCell -drawImage:]: this can be used for 10.6 ~ 10.9.

List View & Column View

  • [TListViewIconAndTextCell -drawIconWithFrame:]: both list view and column view for 10.6 ~ 10.8, but column view for 10.9.
  • [TListNameCellView -updateTagsImageForNode:selected:]: list view for 10.9. This hook point is great enough if you wanna know the node path and can tolerate slight delay of first-time icon replacement. Otherwise, [TDimmableIconImageView -drawRect] would be a better solution showing overlayed icons without delay. I'm still finding a better hook point for getting the node path and fixing the delay issue.

Address Sanitizer on Mac OS X

Prerequisite

  • LLVM 3.4 with Clang and ASan support
    $ brew tap homebrew/versions
    $ brew install --HEAD llvm34 --with-asan --with-clang --with-libcxx
    

How to use Address Sanitizer (ASan)

$ cat use-after-free.c
#include <stdlib.h>
int main()
{
    char *x = (char*)malloc(10 * sizeof(char*));
  free(x);
  return x[5];
}
  
$ clang-3.4 -g -Os -fno-omit-frame-pointer -fsanitize=address -o use-after-free use-after-free.c
  • -fsanitize=address is to switch ASan on
  • -fno-omit-frame-pointer is to get nicer stack traces in error messages

After you execute it, you'll see something as below:

=================================================================
==19561==ERROR: AddressSanitizer: heap-use-after-free on address 0x60700000cc05 at pc 0x10cef6eb2 bp 0x7fff52d094d0 sp 0x7fff52d094c8
READ of size 1 at 0x60700000cc05 thread T0
    #0 0x10cef6eb1 (/path/to/asan/./use-after-free+0x100000eb1)
    #1 0x7fff88bb77e0 (/usr/lib/system/libdyld.dylib+0x27e0)
    #2 0x0
0x60700000cc05 is located 5 bytes inside of 80-byte region [0x60700000cc00,0x60700000cc50)
freed by thread T0 here:
    #0 0x10cf1720e (/usr/local/lib/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.asan_osx_dynamic.dylib+0x1b20e)
    #1 0x10cef6e7a (/path/to/asan/./use-after-free+0x100000e7a)
    #2 0x7fff88bb77e0 (/usr/lib/system/libdyld.dylib+0x27e0)
    #3 0x0
previously allocated by thread T0 here:
    #0 0x10cf17140 (/usr/local/lib/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.asan_osx_dynamic.dylib+0x1b140)
    #1 0x10cef6e6f (/path/to/asan/./use-after-free+0x100000e6f)
    #2 0x7fff88bb77e0 (/usr/lib/system/libdyld.dylib+0x27e0)
    #3 0x0
Shadow bytes around the buggy address:
  0x1c0e00001930: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0e00001940: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0e00001950: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0e00001960: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0e00001970: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x1c0e00001980:[fd]fd fd fd fd fd fd fd fd fd fa fa fa fa fd fd
  0x1c0e00001990: fd fd fd fd fd fd fd fd fa fa fa fa 00 00 00 00
  0x1c0e000019a0: 00 00 00 00 00 fa fa fa fa fa 00 00 00 00 00 00
  0x1c0e000019b0: 00 00 00 fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x1c0e000019c0: 00 fa fa fa fa fa 00 00 00 00 00 00 00 00 00 fa
  0x1c0e000019d0: fa fa fa fa 00 00 00 00 00 00 00 00 00 fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:     fa
  Heap right redzone:    fb
  Freed heap region:     fd
  Stack left redzone:    f1
  Stack mid redzone:     f2
  Stack right redzone:   f3
  Stack partial redzone: f4
  Stack after return:    f5
  Stack use after scope: f8
  Global redzone:        f9
  Global init order:     f6
  Poisoned by user:      f7
  ASan internal:         fe
==19561==ABORTING

You can see much information about the addresses used after free. But I couldn't find actually where are the addresses use-after-free+0x100000eb1 mapping to use-after-free.c. It's easy! Simply specify the ASan symbolizer:

$ ASAN_SYMBOLIZER_PATH=/usr/local/bin/llvm-symbolizer-3.4 ./use-after-free
=================================================================
==29587==ERROR: AddressSanitizer: heap-use-after-free on address 0x60700000cc05 at pc 0x10bf79eb2 bp 0x7fff53c86490 sp 0x7fff53c86488
READ of size 1 at 0x60700000cc05 thread T0
    #0 0x10bf79eb1 in main /path/to/asan/use-after-free.c:7
    #1 0x7fff88bb77e0 in start (/usr/lib/system/libdyld.dylib+0x27e0)
    #2 0x0
0x60700000cc05 is located 5 bytes inside of 80-byte region [0x60700000cc00,0x60700000cc50)
freed by thread T0 here:
    #0 0x10bf9c20e in wrap_free (/usr/local/lib/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.asan_osx_dynamic.dylib+0x1b20e)
    #1 0x10bf79e7a in main /path/to/asan/use-after-free.c:6
    #2 0x7fff88bb77e0 in start (/usr/lib/system/libdyld.dylib+0x27e0)
    #3 0x0
previously allocated by thread T0 here:
    #0 0x10bf9c140 in wrap_malloc (/usr/local/lib/llvm-3.4/lib/clang/3.4/lib/darwin/libclang_rt.asan_osx_dynamic.dylib+0x1b140)
    #1 0x10bf79e6f in main /path/to/asan/use-after-free.c:5
    #2 0x7fff88bb77e0 in start (/usr/lib/system/libdyld.dylib+0x27e0)
    #3 0x0
SUMMARY: AddressSanitizer: heap-use-after-free /path/to/asan/use-after-free.c:7 main
Shadow bytes around the buggy address:
  0x1c0e00001930: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0e00001940: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0e00001950: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0e00001960: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c0e00001970: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x1c0e00001980:[fd]fd fd fd fd fd fd fd fd fd fa fa fa fa fd fd
  0x1c0e00001990: fd fd fd fd fd fd fd fd fa fa fa fa 00 00 00 00
  0x1c0e000019a0: 00 00 00 00 00 fa fa fa fa fa 00 00 00 00 00 00
  0x1c0e000019b0: 00 00 00 fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x1c0e000019c0: 00 fa fa fa fa fa 00 00 00 00 00 00 00 00 00 fa
  0x1c0e000019d0: fa fa fa fa 00 00 00 00 00 00 00 00 00 fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:     fa
  Heap right redzone:    fb
  Freed heap region:     fd
  Stack left redzone:    f1
  Stack mid redzone:     f2
  Stack right redzone:   f3
  Stack partial redzone: f4
  Stack after return:    f5
  Stack use after scope: f8
  Global redzone:        f9
  Global init order:     f6
  Poisoned by user:      f7
  ASan internal:         fe
==29587==ABORTING

Now you know where and how to fix it.

Be a Mac Developer! Tools I Use on Mac.

I'd like to write down what tools I use for Mac/iOS Development. Not sure if I have time to keep it updated, but I'll try my best. :P

  • Deploymate: Deprecated/unsafe API scanner. Mostly, I use it for backward/forward compatibility. You'll need it when newer version of Mac/iOS is coming, or..., when someone ask you to support old OS......
  • Spark Inspector: I use it when I was tuning UI or developing UI components. You can also try another similar tool Reveal.
  • Alcatraz: A Xcode package manager I use for managing most Xcode plugins.
  • XVim: Xcode plugins letting you use Xcode like in VIM. If you are a VIM guy, you almost can't live without it.
  • Homebrew: Package manager on OS X like ports on FreeBSD or APT on Linux. I don't use MacPorts anymore.
  • Hopper: Like IDA pro. It's less expensive and natively runs on Mac.
  • SourceTree: Usually I prefer command line tools. Sometimes I'll use its graph view for surfing on commit history.
  • Dash: To me, it's more like Apple's default document viewer. XD
  • CodeRunner: When I want to write a small test function or some experimental codes in Objective-C, I'll choose CodeRunner. (But I still prefer CLI tools for Python, Ruby, etc. :P)
  • Crashlytics: Crash reporting solution on iOS.
  • Network Link Conditioner: Cool tool provided by Apple for testing your apps under different network conditions. Mattt Thompson gave a excellent guide.

...TBA...

Different Ways to Monitor File Changes

很多應用程式都會需要用到 file monitoring,最常遇到的情況就是偵測 config file 有沒有更動,如果有,就 reload config file 來載入新的設定,這樣的作法可以很方便地修改值,不論是在測試的時候設定 debug information,或是讓使用者開啟一些暗黑功能。常見的方式有:kqueue, fsevent, and GCD (Grand Central Dispatch)。

kqueue

可以參考這篇

fsevent

CDEvents 提供了不錯的 API,但 fsevent 我傾向是用在偵測大量檔案變化的時候使用。asepsis 就有用到 fsevent C API 來實作偵測系統內所有 .DS_Store 檔案的動作 (雖然它是用 C 實作 XD)。

GCD

GCD 的方式是參考這篇修改而來,已經放到 Omazing 裡了。

OMZFileMonitor.h
#import <Foundation/Foundation.h>

@interface OMZFileMonitor : NSObject

+ (instancetype)fileMonitorWithPath:(NSString *)path withUpdateBlock:(void (^)(NSString *path))block;

- (id)initWithPath:(NSString *)path withUpdateBlock:(void (^)(NSString *path))block;

- (void)start;
- (void)stop;

@end
OMZFileMonitor.m
#import "OMZFileMonitor.h"

@interface OMZFileMonitor () {
    dispatch_source_t _source;
}

@property (nonatomic, copy) void (^block)(NSString *path);

- (void)_monitorPath:(NSString *)path withUpdateBlock:(void (^)(NSString *path))block;

@end

@implementation OMZFileMonitor

+ (instancetype)fileMonitorWithPath:(NSString *)path withUpdateBlock:(void (^)(NSString *path))block
{
    return [[self alloc] initWithPath:path withUpdateBlock:block];
}

- (id)initWithPath:(NSString *)path withUpdateBlock:(void (^)(NSString *))block
{
    if (path.length == 0 || block == NULL) return nil;

    if (!(self = [super init])) return nil;

    [self _monitorPath:path withUpdateBlock:block];

    return self;
}

- (void)dealloc
{
    if (_source) {
#if !OS_OBJECT_USE_OBJC
        dispatch_release(_source);
#endif
        _source = NULL;
    }
}

- (void)start
{
    if (_source) dispatch_resume(_source);
}

- (void)stop
{
    if (_source) dispatch_suspend(_source);
}

- (void)_monitorPath:(NSString *)path withUpdateBlock:(void (^)(NSString *path))block
{
    NSParameterAssert(path.length > 0);
    NSParameterAssert(block != NULL);

    self.block = block;

    dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
    int fd = open([path UTF8String], O_EVTONLY);

#if !OS_OBJECT_USE_OBJC
    if (_source) dispatch_release(_source);
#endif
    _source = dispatch_source_create(DISPATCH_SOURCE_TYPE_VNODE,
                                     fd,
                                     DISPATCH_VNODE_DELETE|DISPATCH_VNODE_WRITE|DISPATCH_VNODE_EXTEND|
                                     DISPATCH_VNODE_ATTRIB|DISPATCH_VNODE_LINK |DISPATCH_VNODE_RENAME|
                                     DISPATCH_VNODE_REVOKE,
                                     queue);

    __weak __typeof__(self) weakSelf = self;
    dispatch_source_set_event_handler(_source, ^{
        __strong __typeof__(self) self = weakSelf;

        unsigned long flags = dispatch_source_get_data(_source);
        if (flags & DISPATCH_VNODE_DELETE) {
            dispatch_source_cancel(_source);
            [self _monitorPath:path withUpdateBlock:self.block];
        }

        if (self.block) self.block(path);
    });

    dispatch_source_set_cancel_handler(_source, ^{
        close(fd);
    });
}

@end