Python: xlrd - bad magic number for file header

Problem:
Working in Pylons for a project, I built a tool to allow file uploads and one feature used xlrd to convert Excel files to CSV files. When I went to transform these files using xlrd, I got an error saying: bad magic number for file header.

Solution:
Turned out the file got corrupted as I uploaded it on Windows but did not use a binary flag when storing the file. Adding this flag fixed the issue and allowed xlrd to continue properly.

SSH Tunnel All Network Traffic - Windows 7

Problem:
While working on a some development, there became an issue with the ISP and they hijacked all HTTP requests to return a notice explaining that an issue needed to be resolved. I determined that HTTPS still worked as it uses SSL and therefore would be difficult (and possibly illegal) for them to hijack those requests.

Solution:
SSH is an encrypted connection and therefore was still available to use. Using PuTTY, I set up a connection as a tunnel. Once this was done, I could configure my browsers to use the local socks5 connection to one of our development boxes (make sure you trust your proxy end point as all non-secured traffic will be visible).

Setup a New connection

Enabled compression

Add a local port (dynamic) that will be used for forwarding

Once the port has been added

Note: These are the minimal settings and can be configured further depending on your needs.

This worked (with obvious increased latency due to another node between end points) and allowed me to get access to all HTTP web sites. However, I wanted to take this further for applications like Spotify without configuring each one independently. After some looking around as programs like Widecap, I determined that it would probably be something natively supported by Windows. Here is how I went about setting that.

Click the Internet Icon in the lower right

Click the Network Sharing Center Link

Click on Internet Options Link

Click the Connections tab and then click the LAN settings button

Check "Use a proxy server..." and then click the "Advanced" button

If you are doing 1 socks5 proxy like I am, clear all other addresses and add the local address and forwarded port from earlier.

There are a lot of additional settings, but this should work.

I strongly discourage trying to use this to circumvent employers from knowing of your activities, or for using this to try and remain anonymous online.

Laravel 4: Seeding Large CSV Files Using MySQL Load Data

Problem:
I wanted to quickly populate some tables using some large CSVs (400k+ records) but parsing the files and importing each line was out of the question and doing one mass insert would have used up a lot of memory. The only good solution is using MySQL's LOAD DATA functionality which can import the data within seconds. But figuring out how to do this with Laravel was not easy.

Solution:
When trying to execute the query using DB::statement() I kept getting met with an exception from MySQL saying that I had unbuffered queries active. This displayed with a suggestion of using a PDOStatement. After some searching the internet and trying some things, I found a working solution.

$csv = app_path().DIRECTORY_SEPARATOR."database".DIRECTORY_SEPARATOR."resources".DIRECTORY_SEPARATOR."locations.csv";

$query = sprintf("LOAD DATA INFILE '%s' INTO TABLE locations FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"' ESCAPED BY '\"' LINES TERMINATED BY '\\n' IGNORE 0 LINES (`location_id`, `country`, `region`, `city`, `postal_code`, `lat`, `lng`, `metro_code`, `area_code`)", addslashes($csv));

DB::connection()->getpdo()->exec($query);

Dynamic Relative Date Range

About Dynamic Relative Date-time Range
Dynamic Relative Date-time Range (also known as a Rolling Date Range) is a string formatted representation of a period of time relative to the present time in string form (like in an HTML form). Generally DRDR would be used in reporting environments as it provides a dynamic way to allow a predetermined selection.

Using this form allows coders and non-coders to easily read and write relative ranges. This can also be easily stored in a database for automated reporting.

This syntax requires a language with support similar to PHP's relative date syntax.

General Rules
Caret (^) represents the range separator always separating the starting relative date from the ending relative date. When you begin with a caret or end with a caret, the omitted value is assumed as "now".

Example
last-year^ == last-year^now

The term "ago" can be provided as per PHP relative date syntax documentation to indicate a previous relative date or time.

Example
12-months-ago^now (or) 12-months-ago^

DRDR Example Formats
jan-1-last-year^jan-1
jan-1^now (or) jan-1^
now^next-year-jan-1 (or) ^next-year-jan-1
3-months-ago^
90-days-ago^
last-month^this-month
this-month^
last-week^this-week
this-week^
7-days-ago^
today^
^next-monday
60-seconds-ago^
^next-5-hours
^tomorrow
first-day-of-june^
last-sat-of-July-2008^
^last-day-of-next-month
yesterday-noon^
^monday-next-week
last-hour^


HTML Example
<select name="date-range">
<option name="90-days-ago^now">Last 90 Days</option>
<option name="last-month^this-month">Last Month</option>
<option name="this-month^now">Month-to-Date</option>
<option name="30-days-ago^now">Last 30 Days</option>
<option name="last-week^this-week">Last Week</option>
<option name="this-week^now">Week-to-Date</option>
<option name="7-days-ago^now">Last 7 Days</option>
<option name="today^now">Today</option>
</select>

PHP Implementation
list($start_range, $end_range) = explode('^', $_REQUEST['date-range']);
$start_datetime = strtotime(str_replace('-', ' ', $start_range ?: 'now'));
$end_datetime = strtotime(str_replace('-', ' ', $end_range ?: 'now'));

Future
Currently, I have plans to extends this format to include quarters, but have not yet done so. 

Migrating From Laravel 3 to Laravel 4

I apologize that this page is not well organized. It mainly exists to serve as notes for myself. If you can make sense of this, great!

Changes I had to make to my code when updating from Laravel 3 to Laravel 4:

General
Auth::login(int) => Auth::attempt(array()) or Auth::login(User)
Request::current() => Request::path()
URI::segment(int) => Request::segment(int)
URL::to_route("X") => URL::route("X")
Request::route()->is("X") => Request::is("X")
Form::token() => included by default when calling Form::open() but still accessible
Form (method = POST) => default method when calling Form::open()
Form::open_for_files(...) => Form::open(array('files' => true, ...))
path(...) => storage_path() || public_path || app_path() || base_path()
DS (constant) => DIRECTORY_SEPARATOR
with_errors($x) => withErrors($x)
with_input($x) => withInput($x)
Response::error() => App::abort()
Request::ip() => Request::getClientIp()
DB::query => DB::select, DB::insert, DB::update, DB::delete, DB::statement

Blade
@layout(...) => @extends(...)
@render(...) => @include(...)
@forelse => GONE, use @if(count(...))  and @foreach()  [ this was nice while it lasted :( ]

Eloquent
belongs_to => belongsTo
has_many => hasMany
belongs_to_many => belongsToMany
has_one => hasOne
order_by => orderBy
group_by => groupBy
has_many_and_belongs_to => hasManyAndBelongsTo

Models
set_fieldx($val) {$this->field = $val;} => setFieldxAttribute($val) { $this->attributes['field'] = $val; }
get_fieldx(){return $this->field; } => getFieldxAttribute($val) { return $val; }
where_val() => whereVal()
or_where() => orWhere
where_in => whereIn
where_between => whereBetween (Note: 2nd argument should be array of min, max)
or_where_between => orWhereBetween (Note: 2nd argument should be array of min, max)
or_where_in => orWhereIn
where_not_in => whereNotIn
left_join => leftJoin
::hydrate() => newFromBuilder (read more)

Libraries
You can add app/libraries to the composer autoload json or
create package(s) using workbench and place your code in there.

Error Running PEAR On Windows

Problem:
I went to verify phpunit installation on my windows machine and found that when running pear.bat, I was met with the following:

$ pear.bat
./pear.bat: line 1: @ECHO: command not found
./pear.bat: line 3: REM: command not found
./pear.bat: line 4: REM: command not found
./pear.bat: line 5: REM: command not found
./pear.bat: line 6: syntax error near unexpected token `('
./pear.bat: line 6: `REM Copyright (c) 1997-2010 The Authors'

Solution:
Reading the PEAR Documentation, I found that the program had to be 'run as Administrator'. Doing this allowed me to upgrade pear and then install phpunit.

Automated SMS Messaging From Windows Command Line

Problem:
I saw a contest that requested that people text or call in as many times as they can within a short period of time. As the contest rules didn't state anything about automation, I decided to write a script to do my bidding.

Solution:
The final script was simple enough. By connecting my phone in debug mode to the computer, running android service, and using the following batch script, I was able to begin my bombardment in hopes of winning the contest. This simple code uses an infinite loop and within that calls instructions telling the phone to create a text message, then sends the commands to send the text message by simulating key presses, and then generates a random amount of time between 1 and 30 seconds before doing repeating the process (as an attempt to be at least somewhat reasonable and not fill up my carriers pipes).

:loop
adb shell am start -a android.intent.action.SENDTO -d sms:"+15015551234" --es sms_body "I want to win" --ez exit_on_sent true
adb shell input keyevent 22
adb shell input keyevent 66
SET /A die=30*%random%/32768+1
timeout %die%
goto loop

Now cross my fingers, that my carrier wont care too much... is unlimited texting really unlimited or is that within reasonable usage... btw, what is reasonable usage?

Linux mail text as body shows up as attachment: rkhunter cron

Problem:
I have several servers that run different security automations and email me on set intervals via cron. One of those processes is rkhunter which I run via a shell script using the --cronjob argument. Since I had set these up, the reports would come through but they would also show up as an attachment.

Solution:
After playing with the shell script today, I finally found the problem. rkhunter uses cli colors ( as supported ) when displaying output of certain processes. These colors would get sent in the email and automatically handled by gmail to be an attachment (security procedure I'm guessing). By adding the additional argument
--nocolors, I am now able to send the output as plain text.

#!/bin/sh

( /usr/bin/rkhunter --versioncheck --nocolors
/usr/bin/rkhunter --update --nocolors
/usr/bin/rkhunter --cronjob --report-warnings-only --nocolors
) | mail -s "rkhunter report" [email protected]

Solution - Installed LAMP, PHP works, but website shows a blank page!

Problem:
Recently I've been setting up some servers for web hosting and while install the LAMP stack, ran into a funny issue where the server would return a blank page when I tried to access the PHP website.

Solution:
After checking logs, and playing with settings, I realized I had made a bad assumption. The server hosted PHP version 5.3.3, but the libraries in use required PHP 5.3.7. After installing a newer version of PHP, everything went worked great.

Laravel 3 Documentation: Here it is!

Problem:
With the recent release of Laravel 4, the documentation URLs for Laravel 3 have been completely changed and I haven't been able to find it.

Solution:
Found it:
http://three.laravel.com/docs


Note:
Thanks to JTIndex for the intermediate documentation while laravel domains were being changed around.


Source: laravel.io forums

Using Composer with Laravel 3

Problem:
I found a well-written PayPal REST API that used Composer. My existing project was in Laravel 3. I wanted to make everything work without hacking all my files to bits. But how?

Solution:
By modifying 2 files and adding a new "vendor" directory, I was able to use the composer library as part of my Laravel 3 application.

First, add a "vendor" directory to the root project directory. Second, modify /public/index.php with the following lines BEFORE the launch Laravel directive.

// --------------------------------------------------------------
// Autoload composer vendors.
// --------------------------------------------------------------
require path('composer').'autoload.php';


Now we need to add the path reference being used here into paths.php. I put this after public and before the awesome dragon ASCII art.

// --------------------------------------------------------------
// The path to the composer vendors directory.
// --------------------------------------------------------------
$paths['composer'] = 'vendor';

Now you can place your composer.json in your root directory and run composer update on it. That should do it!



Note: Thanks to http://stackoverflow.com/questions/15205949/class-not-found-with-laravel-3-and-composer for laying some ground work, although that solution still had a bug for me.

Finding a Payment Processor / Gateway - Tons of fees and what you have to know first.

Keep Reading

Most likely, you're reading this because you are either considering accepting credit/debit cards or have been looking for a reliable Payment Processor (PP). Here are some things to help prepare you for what’s to come.

You're A Risk

The PP's job is to assess what kind of risk you are before working with you. They do this by putting you into a risk category. These are some of the different factors involved with determining your risk:

• The sale of digital goods vs tangible goods (if you sell digitally, you're more of a risk)
• Credit history (if your company doesn't have any, then this falls on you personally, if yours isn't good, that’s a risk)
• Amount of time your company has been in business.  If you have no payment history or processing history than you will be in a higher risk category.

Have Good Credit?

Good for you! If you don't, you need a cosigner. Find out what portion of the company the co-signer needs to own, if any. Also, chances are you will undergo a "business model check" where they evaluate your business plan and practices. This check could include references to vendors, suppliers, or other sensitive business relationships.

Fees, Fees, and more Fees

When you start looking into choosing a payment processing service, you will start to notice fees. Lots of fees. These are the fees I'm aware of along with why it's charged and my thoughts on it. 

Gateway Usage Fees: You need a gateway but the fees will depend a lot on your situation, generally you can save some or all of these fees by going with a PP that also provides a gateway or has an existing contract with a gateway company. Not all PPs charge for this.

Setup Fees: These are ridiculous fees charges by some PPs to unsuspecting victims. Setting up your account (entering in your info) is already worth their time (especially if you’re in a contract or paying a monthly fee). I strongly suggest not using any processor that won’t waive these fees. Some PPs will say these fees are due to application processing (see Application Fees below). Many PPs do not charge for this.

Gateway Setup Fees: See Setup Fees above.

Merchant Account Setup Fees: See Setup Fees above.

Online Check Processing Setup Fees: See Setup Fees above.

Application Fees: These fees are associated with the work required to process your application, do a background check, evaluate your business, etc. Not all PPs charge for this.

Statement Fees: These are fees assessed for generating your statements each month and possibly mailing them to you. Not all PPs charge for this.

Monthly Fees: Almost all processors will charge a monthly fee to use their service. For example, PayPal will charge you $30 each month for their Pro account. If you are in a contract, you will be required to pay monthly fees throughout the duration of the contract.

Minimum Transaction Fees: This fee is nearly invisible until it becomes a problem. This happens when you don't pay the PP enough money in fees for transactions processed (see Transaction Fees below). For example, if you don't meet a specific total amount (e.g. $25) in fees for the month, they could charge you an additional $25. This is generally always negotiable. If you choose a merchant service provider who charges these fees, get them as low as you can or waived completely. Not all PPs charge for this.

Under Limit Fee: This fee may sometimes be considered the same thing as a Minimum Transaction Fee. This fee is applied when you don't reach a minimum transaction amount in a given period. This fee on average is about 1% - 5%. Not all PPs charge this.

Over Limit Fee: This fee is what is sounds like; a fee applied when you go over your allowed transaction limit in a given period. On average this is about 1%-5%. Not all PPs charge this.

PCI Compliance Fees: These fees are essentially a charge added for the effort required in making the service PCI compliant. Not all PPs charge for this.

Charge-back Fees: These are almost unavoidable. Fortunately, if your doing business ethically and correctly, it shouldn't be much of a problem. Some people might get this confused as a fee for refunding money to your customers (see Refund Fee below), but these are fees associated with when a customer calls the credit card company demanding a payment reversal and their dispute has to be managed.

Charge-back Monitoring Fees: Not sure

Terminal Fees (or Virtual Terminal): Some Merchant Service Providers will add an additional charge to use a terminal service. This is generally a website that allows you to manually type in credit cards for customers who call over the phone or are not in person. Not all PPs charge for this.

Card Issuer Fees: The company who issued the Credit or Debit card (i.e. VISA, AMEX, Discover, etc.) adds a charge to process your payment. Some Processors absorb this cost (PayPal), while others pass it onto you (FastCharge). 

Transaction Fees: All processors will charge a fee per transaction. This may be a flat rate, a percentage of the transaction, or both. Some companies may also have different transaction rates depending on if the customer uses a credit card or debit card. For example, PayPal requires that you pay 2.9% + $0.30 per transaction regardless of the card. Elavon charges 2% per transaction per credit card and 1.6% for a debit card. When you talk to PPs, verify that the quoted transaction fees are for internet-based transactions.

Recurring Billing Fees: These are fees charged for offering a service of recurring billing. For example, PayPal charges an additional $10 per month, whereas FastCharge offers this service as no additional cost. Not all PPs charge for this. 

Batching Fees: You may want to process batches of transactions at the end of the day. Some PPs may add an additional charge for this.

Cancellation Fees: If you happen to enter into a contract, you can expect that you will assess a penalty for ending that contract early. For example, ending a 3-year contract with Elavon early costs you $250.

Payment Transfer Fees: If you want access to your money, you may have to pay for it. Inquire with potential PPs about any payment transfer fees or limitations on how many times you can transfer during a given time period.

Unverified Credit Card Fee: Sometimes processors will require a higher transaction rate if the credit card information provided doesn't include other identifying information like a zip code. Processors like Elavon may charge an additional 1.5-3% to your existing transaction fees.

Dues and Assessments: These may or may not be the same as Transfer Fees. Inquire with your PP about any D&A fees.

Programming Charge: I haven’t been able to get anyone to explain what this charge is for. My theory is that this began with POS terminals and devices. Avoid PPs who charges for this.

Annual Fees: This is a fee you are required to pay annually to be part of their network. Most PPs don’t charge this.

Debit Network Access Fee: You may be required to pay an additional fee for processing debit cards. Ask your potential PP about debit card processing fees. Most PPs don’t charge this.

Address Verification Fee: This is a fee some PPs may charge for using their Address Verification service. Some processors may require this service is used. Not all PPs offer or charge for this or offer this service.

Refund Fee: A small charge rarely charged by PPs when you issue a refund to a client. I would stay away from any company with a refund fee. Not many PPs charge for this.

Funding Fee: I'm not sure what this fee is supposed to be for. But it may be about 1% daily. Not many PPs charge for this.

Periodic Rate Review: Sometimes you will be required to pay a fee to have your rates reviewed and see if you qualify for lower rates and higher limits. Some companies may charge $50 or more to handle your request. Not all PPs charge for this.

Shopping Cart Functionality: This is a provided shopping cart functionality. Some PPs may charge up to $50 each month for this feature. This will be according to your needs and budget. Not all PPs offer this. Some PPs do not charge additional fees for this.

Secure Payment Forms: Not sure. Some PPs may charge up to $30 each month for this service. Not all PPs offer this. Some PPs do not charge additional fees for this.

Tokenization: This service allows for securely storing sensitive data and returning an token to allow utilizing stored data. For example, some processors allow you to securely store credit cards and process on those credit cards without the needing the card present and without allowing that information to be retrieved in plain format from storage. Not all PPs charge for this.

3DES Encryption: 3DES or Triple Data Encryption Standard is a method used for encrypting data stored on PP servers. Not all PPs charge for this.

Proxy Blocking: Known proxies can be used to maliciously attack your site and attempt to cause harm to your billing process. However uncommon, proxy blocking is one way to help alleviate this trouble. This can be provided at an extra cost, or included with your base program.

IP Blocking: IP blocking allows you to protect your service from malicious IP addresses.

CVV/CVV2 Verification: This fee can apply in order for a processor to verify

Advanced Fraud Screening: In the event that someone uses a fraudulent card on your web site, this service will help to prevent this from happening which can reduce ca

Country Blocking: Just as one might want to block IP and Proxys, you can sometimes also choose to block countries.

Sub-accounts / Additional Users: Some processors will add an additional charge if you want to add additional users or sub accounts to your primary account.

Domain Blocking: This is very similar to IP, Proxy, and Country blocking.

Brute Force Prevention: Malicious people online can use scripts to automate attacks against your site and services your site utilizes. Many PPs offer this, some charge additional service fees.

Other Things to Consider

Negotiate: Most of the fees listed are flexible, and (whether they admit it or not) they can be dropped or modified. Don't be shy about pivoting one processor against another. They would much rather have your business and not get all their fees than lose you completely.

Security Reserves: Sometimes companies will require you to keep a certain amount in their account at all times. This amount will be bigger for high risk companies.

Contracts: A lot of processors suggest that you don't get locked into a contract. Elavon offers a standard 3-year contract. Personally, I avoid contracts as it allows the provider to get lazy in their support of my product and their interest in me as a customer.

Restricted Countries: Some PPs only accept payments from people within the United States. Others have a list of specific countries from which they accepts customers or have no restrictions.

Restricted Products: Some companies let you sell only physical products or only intangible products, which can be downloaded or emailed and don't need to be shipped. Others have no restriction on what you can sell through them. 

Fraud Screening Charges: This is generally an optional service which will help prevent fraudulent charges and charge-backs.

Underwriting: Find out if they are using an in-house underwriter or sending it out. This may affect your getting an account.

Credit Checks: Find out when your credit is pulled. They will see what other processors have looked at your credit. The more processors that show up, the higher of a risk you appear.

Approval Time: Find out how long it takes to get approved. Typically, PPs approval time in anywhere from 24-72 hours to 1-3 weeks.

Maximum processing limits per month: Some processors will limit how much money in transactions you can do per month. This is based on your level of risk and your merchant history. For new businesses, it may take up to 2-3 years to establish a good merchant history. (See Over Limit Fees above for more on related fees.)

Micro-payments: If you are planning to handle many small transactions, you will want to look for a provider who allows for Micro-payments. These rates generally are going to save you money in the long run if you have high volume. PayPal offers this option for a transaction rate of $0.05 + 5%.

Alternate payment methods: If you are planning on taking other forms of payment other than credit or debit cards, find out if the processor allows for wire transfer or ACH processing.

Bookkeeping: Find out if their software will integrate with your bookkeeping software. This can make managing finances much easier in the long run.

Development API: You may want to have a programmer integrate your own shopping cart or handle charges in another way. In these cases, you want to make sure your processor has an API available.

Final Note

I am not compensated by any company referenced to on this page either directly or indirectly. Neither am I the authority or official representative of these companies. The prices listed were pulled from the company websites (2013) or explained verbally over the phone. The opinions on this page are purely those of the author and do not reflect any official opinion of CorrLabs, LLC. For exact fee amounts or official reasons for fees, please contact the Payment Processor directly.

XBMC Addon Development - Updated Zip file doesn't update addon issue

Problem:
While working on an XBMC addon, I got my addon to install properly and then decided to make some changes. I Uninstalled the addon and tried to install the new Zip file but the old addon showed back up and none of my changes existed.

Solution:
I'm still not sure why I can't upload a new Zip file with an update, but I found a better way to development the addon anyway. By placing the folder with my addon into the {XBMC Install location}/addons folder, I was able to restart XBMC and have my changes take effect immediately.

I hope at some point I can understand why updating via a Zip file wasn't working, but for now, I'm just going to develop in the addons folder. If you know why it wasn't working, leave a comment below!

XBMC: ERROR: Unable to create application. Exiting - Fix On Windows 7

Problem:
In my spare time I've been writing a python script addon for XBMC. While developing I was tailing the log file, but trying to restart the application, I got the following error message:

ERROR: Unable to create application. Exiting

Solution:
It turned out that for me, this error was caused by keeping the tail open on the log file when trying to restart the XBMC application. Closing the tail and reopening each time I launched the application made the application launch correctly. If you're not using a tail, the same error may occur when files used by XBMC are locked or being accesses by left over threads or other applications.

Laravel 3: Unlimited routing arguments

Problem:
I was working on a project in Laravel when I realized I really wanted to have 1 route rule for a specific controller but be able to allow a lot of different arguments.

Solution:
I found that I could use func_get_args() in conjunction with the static Controller::call() method to handle an unlimited number of arguments to the receiving controller method.

Route::any('users/(:any?)/(:any?)/(:any?)', array(
        'as' => 'users', 
        function(){
            $args = func_get_args();
            if (empty($args)){
                $action = 'index';
            }else{
                $action = array_shift($args);
            }
            return Controller::call("users@{$action}", $args);
        }
    ));

There is one caveat to this solution, and that is that you have to specify all the possible captures in the Route URL rule although extra captures are ignored.

Adobe AIR: Getting HTML Source on MX HTML Component

Problem:
I decided to write a tool that would aggregate data from web sites as one surfs the web, kind of like a browser with some extra functionality. It soon became apparent that the <mx:HTML> component didn't have a simple "source" property where I could see all the HTML from the website.

Solution:
It turned out to be a simple task. The HTML component has a property which allows you to access the HTMLLoader object associated with the component. The HTMLLoader object has direct DOM access using Psuedo DOM objects (basically basic objects with a couple preset properties called Nodes and NodeLists). Many of the JavaScript dom functions exist in this representation and will work just fine.

For example:
getElementById
getElementsByClassName
getElementsByTagName

Using the last one along with the innerHTML property, we can get the source of the body element.

html_component.htmlLoader.window.document.getElementsByTagName('body')[0].innerHTML;

I'm sure you could get the entire document source if you wanted to examine the page head, etc. But these weren't useful for me.

Good luck!

Laravel Routes: Missing argument 1 for {closure}()

Problem:
While trying to develop quickly I set up a routing rule for Laravel that I thought was correct. But when I tried to access the page, I was met by this error:

Missing argument 1 for {closure}()

My Routing Rule looked like this:

Route::any('dashboard', array(     
    'before' => 'auth',     
    'as'     => 'base',      
    function($action){         
        return Controller::call("user@index");     
}));

Solution:

This was an oversight on my part, the closure used in the route expected an argument which would be the information obtained by any shortcuts [(:any),(:num),etc.] or regex rules. Since this particular route didn't have any captured portions of the URL, the closure also expected no arguments. My working route looks like this with the argument removed:

Route::any('dashboard', array(

    'before' => 'auth',

    'as'     => 'base', 

    function(){

        return Controller::call("user@index");

    }

));

MySQL: Prepared statement needs to be re-prepared

Problem:
While developing a web site on my development service provided by Dreamhost, I found that I would occasionally (and randomly) run into the following error:

General error: 1615 Prepared statement needs to be re-prepared

I began searching for a solution thinking it might be something with the Laravel framework. Then I found this page about a MySQL bug.

Solution:
Ultimately it appears that the best solution is to force the table cache to renew in mysql (more information about mysql repreperation), but if that is not an option, you can try forcing the table optimization or defragmentation for the table causing the issue.

This is understandably a work-around which is not idea for a production server. In the case of a production server, you can try modifying any of the following database settings:

table_open_cache 128=>16384
table_definition_cache 1024=>16384
tmp_table_size 32M=>64M
join_buffer_size 256k=>512k

If you have any more information on this issue, please post a comment below!

BitBucket: info/refs not found: did you run git update-server-info on the server?

Problem: 
I set up a new repo on bitbucket.org for a project I've been working on and once it was ready for me to add my code, I kept getting the message:

info/refs not found: did you run git update-server-info on the server?

Solution: 
After destroying and creating my .git folder over and over trying to verify my remote path and that I had done the same steps as I had done many times before, it finally clicked what the problem might be. When I had names the repo on BitBucket.org, I had used a hyphen in the name. I changed the name to have a space instead of a hyphen and changed it enough that BitBucket accepted it as a new name. Then I repeated the steps as before and it worked as expected!

Now there are many reasons why you might get this message but hopefully this gives you a possible solution.

Good luck!

Flash Builder: Building Crashes Player - Vector Issue

Problem:
While working with some vectors in AS3, I was in a hurry and typed out my vectors like this:

private var image_types:Vector.<String> = new Vector.<String>("png",'jpg','jpeg','gif');

This caused the Flash Player to crash without giving an error and needless to say, I had to reverse my steps until I figured out it was this Vector that was the problem.

Solution:
In the code I displayed above, I assumed that I could define some elements on Vector construction like you can with the Array class. This assumption is what caused the problem and ultimately a bug that isn't isn't caught when compiling the code down, but unexpectedly caught on execution. The Vector class doesn't allow you to pass list items into the constructor, but instead allows you to pass an uint and a Boolean which tell it what length to have and if it should be fixed.

The solution was simple enough though, using the Vector global function, I was able to modify my code slightly and have it compile and execute as expected.

private var image_types:Vector.<String> = new Vector.<String>(["png",'jpg','jpeg','gif']);

AS3 : Setting the Mouse Cursor

Problem:
Sometimes it can be difficult to use custom cursors in AS3.

Solutions: 
The Mouse singleton class makes this really easy and fluid. If you compare a side-by-side of a SWF using the old techniques vs the Mouse singleton class, you will notice the difference.

Example: Setting the cursor to an image.


// Embed a graphic for the custom cursor
[Embed(source="assets/cursor.png")]
[Bindable]
private var DrawCursor:Class;

// Setup the cursorData object
private var cursorData:MouseCursorData = new MouseCursorData();


// First we setup a vector to handle the bitmapdata objects (you would use more than 1 if you wanted an animated cursor).
var drawCursorData:Vector.<BitmapData> = new Vector.<BitmapData>();
// Now we need to get the bitmap from our embedded graphic
var drawCursor:Bitmap = new DrawCursor();
// now populate the bitmapdata vector with our graphic bitmap data
drawCursorData[0] = drawCursor.bitmapData;
// the default line-up with your cursor's pointer is 0,0 - but if you need to adjust it, you can set the hotSpot to a new x,y
cursorData.hotSpot = new Point(2,12)
// set the cursordata to contain the bitmapdata vector
cursorData.data = drawCursorData;
// register the cursor so you can use it later
Mouse.registerCursor("drawCursor", cursorData);

// When you're ready, set the mouse cursor using the cursor property to the registered name.
Mouse.cursor = "drawCursor";

// And when you're all done, you can restore the cursor back using the static property of the MouseCursor class
Mouse.cursor = MouseCursor.AUTO;

Note: There are several default cursors available in the MouseCursor class like Arrow, Button, Hand, and IBeam.

Good Luck!

AS3: Custom Components Flicker / Mouse Event Bubbling

Problem:
While working on a project, I added some button components within another component and then had the buttons hide unless the mouse was over the entire component. Each time I would move the mouse around over the component, the buttons would show and hide quickly causing a flickering.

Solution:
I had forgotten that there is a different between MouseEvent.MOUSE_OVER and MouseEvent.ROLL_OVER. Using MOUSE_OVER defaults the events to bubble which causes other listening items to respond to that event whereas ROLL_OVER doesn't bubble the event.

Raspberry Pi: Remote Canon Camera Viewing and Control

This is my ongoing wireless video streaming project from my T2i using Raspberry Pi.

NOTE: I will organize this better when I have more time, I promise.

One possible option: http://www.auvidea.com/index.php/theme-styles/2013-06-22-21-23-34/raspberry-pi-hdmi-in
This would allow an HDMI in (non-encrypted) and also would provide a wifi system which could potentially encompass everything I wanted to do and do it better.

Another non-pi options: http://www.auvidea.com/index.php/theme-styles/2013-06-22-21-23-36/encoder-e100

Interesting Reading: https://plus.google.com/+RyanMatthews/posts/btdqMZqCzEA

In the Beginning:
I though it would be really nice to have some way to view my live video feed from my Canon T2i remotely from a mobile phone or tablet computer. It has crossed my mind that using this setup, I could potentially remote control the camera using PTP (Photo Transfer Protocol) and MTP (Media Transfer Protocol). I realize there will be a need to a computer to handle mediation between the camera and the remote viewing device. Only one other commercial device has been found which is for serious cameras and sells for $1,500. I'd like to do it for less than $100 using the Raspberry PI.

Some similar projects in regards to still photography acquisition and transfer:
http://islandinthenet.com/2012/08/23/hdr-photography-with-raspberry-pi-and-gphoto2/
http://davidhunt.ie/?p=2641

References:
http://blog.waynehartman.com/archive/2009/01/07/edsdk-to-remote-control-your-canon-camera.aspx
http://www.graphics.cornell.edu/~westin/canon/index.html (powershot)
http://gphoto.org/doc/manual/quickstart.html#using-gtkam 
http://islandinthenet.com/2012/08/23/hdr-photography-with-raspberry-pi-and-gphoto2/

10/01/2012 (or so) 
I purchased the Raspberry PI model B (256MB) (512 wasn't available yet) online from Allied Electronics for $35 plus about $8 shipping.

10/05/2012 (or so) 
I determined the design and hardware requirements if this is to be obtainable. The Canon camera has an EDSDK which allows a developer to write programs that will interface with the canon Firmware and hardware (using APIs). The EDSDK however only appears to be supported on Mac and Windows (Linux not officially supported). One of these controls is the live view data. Using a USB connection between RPi (USB2.0) and the Canon (USB2.0) will allow the Pi to establish a connection with the camera to control it. A WiFi dongle will be used to enable the Pi to be a WiFi hotspot. Pi will use something similar to VLC or ffserver to stream image data over a wireless protocol to the listening device. The device will have a custom built solution to allow it to render out the media stream. Using a socket (or even RESTfully, commands could be sent back to the Pi to change focus, contrast, etc).

10/19/2012
Rasberry Pi shows up but I can't do anything really for it. Bought an SD card ( $8 / 16 GB Class 10 like this one) and an adapter from HDMI male to DVI female (about $5; who knows if this will work).

10/24/2012 
Adapter and SD card arrive, switched SD card with wife for 8 GB Class 4. Adapter still needs to be tested while OS formatted SD is in the Pi.

10/25/2012
Did some testing with the PI and research about Canon cameras and USB control. Applied and obtained the EDSDK from Canon. SDK appears to support control for the live view along with histogram data, etc. Pi boots up wheezy raspbian and is working great, still haven't tried it with the HDMI/DVI adapter. It's running quick though. Much better it seems then it did on the emulated version.

Preparing myself for adventures in C. Planning to try to remote access Camera data via Windows 7 desktop to ensure it will work and that I can get the data I want from the Canon. If all goes well, I will then move the programming onto the Pi and begin networking the hotspot and streaming services.

10/30/2012
Been thinking a lot about how to get the RPI setup to work better and did some more research into gphoto2. Found out Gphoto2 will allow the live view to be captured on some camera. Decided to setup a fedora box as a sandbox for testing this. VirtualBox fedora wouldn't recognize the USB devices (I might not have had it setup right, and I'd been looking for an excuse to finish setting up my old Dell with Linux).

11/01/2012
I began doing some testing now that I have a sandbox setup. Gphoto2 successfully recognizes my camera and can control it over USB. I am able to take the capture data and have it send to stdout. With ffmpeg I have figured out how to accept the stream and convert it for ffserver to send over the network.

My command is like this: 
ffserver -f /etc/ffserver.conf | gphoto2 --stdout --capture-movie | ffmpeg -f mjpeg -r 2 -i - http://localhost:81/canon.ffm

This appears to be working and ffserver is reachable over my local network. However, when I go to open the native stream in the browser ( which supports playing mjpeg streams) I get the error: This feed is already being received. This was a simple solution.


My ffserver.conf file looks like this:

Port 80
BindAddress 0.0.0.0
MaxClients 10
MaxBandwidth 50000
NoDaemon
<Feed canon.ffm>
file /tmp/canon.ffm
FileMaxSize 10M
</Feed>
<Stream canon.mjpeg>
Feed canon.ffm
Format mjpeg
VideoSize 640x480
VideoFrameRate 10
VideoBitRate 2000
</Stream>

As you can see, there is no conversion happening other than re-sizing the frame size. These setting provide a stream with the following attributes:

• fps 13
• bitrate 233.7kbitz/s

I assume changing my ffserver.conf will allow me to change the output format to be compressed, and allow me to up my frame rate.

11/03/12
Today I finally saw my camera stream over the network in real time. This attests that it could definitely be controlled and viewed a remote device on a wireless network. I'm about to buy a USB WiFi dongle so that I can begin porting this functionality to the Raspberry PI. I'm really excited to be seeing this working so well. I ended up buying a case for the PI for about $10, just to prevent damage from static electricity and dirt. Who knows when this will arrive though. The stream today has some lag, but I doubt it is from a wired network, so I'm going to look into how to optimize the image transfer. Maybe I will add some hooks to allow the client to control the output stream. Currently it has the following specs:

• MJPEG uncompressed output
• fps=15
• q=0
• bitrate 292.6 kbits/s

I'm going to optimize the output format to be compressed and see if I can get a 720 resolution with decent frame rate.

The stats page from ffserver look like this:

Screen capture of the video streaming

h264 links:
http://www.dexmac.com/index.php/how-to/74-streaming-with-ffserver
http://ffmpeg.org/sample.html
https://lists.libav.org/pipermail/ffserver-user/2011-January/000248.html


USB Dongles Options:
http://elinux.org/RPi_VerifiedPeripherals#Working_USB_Wifi_Adapters
http://www.edimax.co.uk/en/produce_detail.php?pd_id=328&pl1_id=1&pl2_id=44
http://dx.com/p/ultra-mini-nano-usb-2-0-802-11n-150mbps-wifi-wlan-wireless-network-adapter-48166?item=1&Utm_rid=24958662&Utm_source=affiliate
http://thepihut.com/products/usb-wifi-adapter-for-the-raspberry-pi
http://www.buyraspberrypi.com.au/raspberry-pi-802-11bgn-usb-wireless-dongle/ (w/ antenna)

11/16/2012
Been very busy and haven't been able to work on this much. Finally at 11 PM on Friday night, I am making some time (to take a break from AMS suffering).

I determined that my goals from before were somewhat mixed, I wanted to find a good output format and have low latency with very little processing power. To make this simpler I have divided the goals hoping I can eliminate type conversion all together.

My initial thought for achieving fairly low latency and still keeping resolution higher is to try and implement 3 things:

1. Decent but not incredible frame rate, I think if I can get it to be under 18, that will help and still be decent
2. Try to interlace the video frames to send less data overall
3. Implement a loose packet protocol UDP (of course)

Lowering the frame rate actually hasn't added much because ffmpeg wont let me get to more than 17 or so. Also, I note that my bit-rate is incredibly high (3857 kbits/s). My resolution is 640x480.

11/17/2012
Tried to setup the same situation on the PI but ran into issues getting the same stream to get captures and transmitted. Going to install everything from source if I can to see if I am running into issues because of the packages available through the PI repos.

------ REVISITED ------

07/25/2016

After getting the Pi 3, I decided to try this again. This time I plan to take better notes.

* Installed raspbian
* Expanded to use entire sd card
* Setup my wifi by updating wpa_suppliment to connect to my router
* run all updates
* build latest gphoto2 (remove any existing, install deps)
       - $ sudo apt-get purge gphoto2
    - $ sudo apt-get install libpopt-dev libltdl-dev libusb-dev libusb-1.0-0-dev -y

    - Download the tar.gz for libgphoto2 and gphoto2 (i.e. 2.5.10) from sourceforge

       = Once they are downloaded, tar zxvf on each download, cd into each extracted dir and run (libgphoto2 must be done first):

       = $ ./configure --prefix=/usr
       = $ make
       = $ sudo make install
* downloaded https://fattylewis.com/latest-ffmpeg-compiled-for-rasbian/ (bui
* (optional, if desired, sym link the binaries in the ffmpeg download to some place on  your path (i.e. /usr/local/bin)
* Ran into an issue where gphoto2 would not detect my canon 550D on RPi3. Halting for now

Next:
* Solve the gphoto2 issue
* Install HostAPD - https://www.raspberrypi.org/forums/viewtopic.php?f=36&t=138550

jQuiz: A professional quiz plugin for jQuery

jQuiz for jQuery on Github

A little while ago I wrote a quiz component for jQuery. I wanted to make it professional and easy to use. It features: multiple choice, multiple response, fill in the blank, essay, ordered list, and now hot spots! The example that comes with the plugin is written in PHP and demonstrates the AJAX/PHP interface with a SQL database template file.

Download now on Github

Programming: Calculating Ongoing/Moving Average - Average Over Time

Problem:
I am writing a platform that will track ratings of people over time and display an average star rating per person. It dawned on me that there should be a solution out there, so I went looking. If you're not familiar with calculating an average, you would normally do a mean average like this:

(n1 + n2 + n3 + n4) / n_count

Like this:

(1 + 2 + 3 + 4) / 4     // The 4 as the divisor because we added 4 numbers together.

But this doesn't work very efficiently when your programming an application that will constantly be displaying the user's average, not to mention having to do the query to get all the values and run the calculation every time would mean using a lot more in server resources.

Solution:
Use a weighted average over time (also sometimes called an ongoing average, rolling average, running average, or moving average). You will still need a couple reference points in order to do this, but you will find it will be much simpler on both the calculation and the resource usage. This will only need to be calculated when input numbers affecting the weighted average have changed (been added, removed, or modified).

new_avg = (existing_avg * existing_number_of_ratings + new_rating) / (existing_number_of_ratings + 1)

Now new_avg will be the current weighted average for that user. Just be sure to store the existing number of ratings so you can compute this again later.

Note: Mean average and weighted averages wont always produce the same result.

PostgreSQL fails to install on Windows

Problem:
Needing a database with a little more flexibility than MySQL, I decided to give PostgreSQL Server a try. While installing the latest version on Windows 7 (64 bit), I ran into an issue where the installer would freeze after entering the port to use. Originally I thought my download was corrupt, so I downloaded it again and tried an older version, but all of them would quit right after asking for the port.

Solution:
I still don't know exactly why the freeze happened, but upon restarting my computer, the installer completed successfully. My guess is that something I had uninstalled earlier in the day causing something in the system to need a reset.

Yes, another one of those, "Restart your computer" fixes, but it worked.

Sphinx Search - Error 1067: The process terminated unexpectedly.

Problem:

Installing Sphinx Search on my computer to do some testing, I bypassed the heavy documentation in hopes of a quick start-up. Instead, on trying to start the Sphinx Search on Windows, I was met by an error reading:

Windows could not start the SphinxSearch service on Local Computer.
Error 1067: The process terminated unexpectedly.

Solution:

This is generally caused by the configuration file being setup incorrectly. Ensure you have configured the conf file correctly before starting the Sphinx Search service. To determine exact errors, trying running the search daemon from the sphinx bin directory.

C:\sphinx\bin> searchd
C:\sphinx\bin>indexer --rotate --all

If you're still not sure how to get going with Sphinx Search, check out this fantastic video walk through:

PHPActiveRecord Fix to Support Foreign Characters Charset

Problem: 
While developing a multilingual application for a client using PHPActiveRecord, I noticed that foreign characters were not getting read properly from the database due to an incorrect charset.

Solution:
The problem is two fold. First, the database connection used needs to be set to have a UTF8 charset. Second, the documentation for UTF8 charset support for PHPActiveRecord is wrong.

From PHPActiveRecord documentation:

1 $config->set_connections(array(
2   'development' => 'mysql://user:pass@localhost/mydb;charset=utf8')
3 );

The correct version / the fix:

1 $config->set_connections(array(
2   'development' => 'mysql://user:pass@localhost/mydb?charset=utf8')
3 );

Notice that the end has a ? before charset and NOT a semi-colon. This is because this path is parsed using PHP parse_url which doesn't recognize the content following the semi-colon as a query.