The Little Array That Could

To Each His Own Array

Intro

Hello everyone, and welcome to my new blog! It couldn’t be a more opportune time than the middle of the summer to start up a new programming site! I would like to start off by saying that I am very excited to share some of my great experiences I’ve had in the few years that I’ve been programming, and start some interesting discussions with other bloggers! Now, this being my first post, I would also like to advise you to sit tight for this, because you’re in for a wild ride!

The Making of PowArray

Let me backtrack a bit and pick up just before from where I ended off on my about me page. So at the peak of last year’s fateful summer, I was at my desktop computer, frantically coding away at my keyboard in the blistering heat, determined to solve Project Euler #16 without any Big Integer class…

View original post 644 more words

PE #20

To Each His Own Array

Intro and Problem Statement

Hello again everybody, this is the second follow up to my post about my Arbitrary Precision Calculator, PowArray. In this post I will be revealing and discussing my solution to Project Euler problem #20 (the basis for my Factorial calculator in PowArray).

Problem #20 states the following:

Factorial digit sum

n! means n (n − 1) … 3 2 1

For example, 10! = 10 × 9 × … 3 2 1 = 3628800,
and the sum of the digits in the number 10! is 3 + 6 + 2 + 8 + 8 + 0 + 0 = 27.

Find the sum of the digits in the number 100!

A Foreword

I’ll limit the number of words in this post in favor of code, as the ideas are pretty similar and not worth reiterating verbatim.

Before jumping right into the solution, it is important to…

View original post 354 more words

Zidisha is hiring a remote engineer

Zidisha: P2P Microfinance

homepage

We’re looking for a remote engineer to help us:

  • Build out our nonprofit P2P microlending platform using the PHP Laravel framework
  • Bring opportunity to some of the world’s poorest places by connecting them to a marketplace that’s independent of geography
  • Develop the world’s first direct P2P microlending community – a model that is revolutionizing access to finance in developing countries

Our technology stack: PHP, Laravel Framework, Propel ORM, Elasticsearch, Ansible, Vagrant, Composer, Git, Github, PHPUnitTest, Twitter Bootstrap, Rackspace Cloud, Nginx, jQuery, many third-party APIs (Facebook, Google, Stripe, Paypal, Telerivet, SendWithUs, SiftScience, Geoip2, Phpexcel, Supremenewmedia, guzzelhttp, laravel-phone).

What You’ll Do

As our sole engineering hire (for now), here are some things you might do:

  • Refactor our Laravel 5 codebase
  • Add and update unit tests
  • Optimize queues (Beanstalkd) and database queries to handle a rapidly growing transaction volume
  • User experience improvements: smarter loan project search results, customizable email digests for lenders, loan balance queries via SMS for borrowers, etc.
  • Work with data scientists to integrate credit risk prediction…

View original post 636 more words

Add emoji support to any Html input

One small but very interesting capability which we provide to our customer is the ability to create a campaign ( Push, in-app, email, web) with Emoji support.

Emoji though a very small feature but makes app developers and marketers connect with their users in an unique and fun way.(technically its called Visual-first strategy).

Coming to technical aspect now.

1.What have we used?

          We have used an open source Github repo. Please read it’s Readme.md file. It is self-explanatory. And the respective issues pull request is here

2.Few Issues which had to be tackled

  • First issue is a very famous Chrome Browser issue! Some Emojis Don’t render well in the input box

         Solution!

          We just skipped those emojis while rendering! Later, when chrome updates we can                 remove this particular snippet.

         Note : Different browser renders emojis differently!

           Here is a small snippet of code which does this…(you will find this in <repo>/lib/js/jquery.emojiarea.js in Emojimenu.prototype.load function)

Screen Shot 2016-04-03 at 11.54.04 AM.png

2. Second issue is that while rendering emoji when you use h1 tag, b tag
or apply <“style= font-weight : bold”> to the element  where
you have added class = emoji-container-picker , emoji is rendered invisible.

Solution!

             One fix we got is, you can wrap each emoji in with style= font-weight: normal .
Else you can separately render text in bold and emoji normally that also helps.

Thats it! Thanks for reading.

Some of the examples which some our customers use via CleverTap.

12
In-app Notification to engage users based on their actions and inactions

Screen Shot 2016-04-02 at 8.51.18 PM
Push Notification to engage users based on their actions and inactions

Thank-You!

How to Remove Duplicates in Large Datasets

 

Dealing with large datasets is often daunting. With limited computing resources, particularly memory, it can be challenging to perform even basic tasks like counting distinct elements, membership check, filtering duplicate elements, finding minimum, maximum, top-n elements, or set operations like union, intersection, similarity and so on.

Probabilistic data structures to the rescue

Probabilistic data structures can come in pretty handy in these cases, in that they dramatically reduce memory requirements, while still providing acceptable accuracy. Moreover, you get time efficiencies, as lookups (and adds) rely on multiple independent hash functions, which can be parallelized. We use structures like Bloom filters, MinHash, Count-min sketch, HyperLogLog extensively to solve a variety of problems. One fairly straightforward example is presented below.

The problem

We manage mobile push notifications for our customers, and one of the things we need to guard against is sending multiple notifications to the same user for the same campaign. Push notifications are routed to individual devices/users based on push notification tokens generated by the mobile platforms. Because of their size (anywhere from 32b to 4kb), it’s non-performant for us to index push tokens or use them as the primary user key.

On certain mobile platforms, when a user uninstalls and subsequently re-installs the same app, we lose our primary user key and create a new user profile for that device. Typically, in that case, the mobile platform will generate a new push notification token for that user on the reinstall. However, that is not always guaranteed. So, in a small number of cases we can end up with multiple user records in our system having the same push notification token.

As a result, to prevent sending multiple notifications to the same user for the same campaign, we need to filter for a relatively small number of duplicate push tokens from a total dataset that runs from hundreds of millions to billions of records. To give you a sense of proportion, the memory required to filter just 100 Million push tokens is 100M * 256 = 25 GB!

The solution – Bloom filter

The idea is very simple.

1.Allocate a bit array of size m
2.Choose k independent hash functions h_i(x) whose range is [ 0 .. m-1 ]
3.For each data element, compute hashes and turn on bits

1

4.For membership query q , apply hashes and check if all the corresponding bits are ‘on’

Note that bits might be turned ‘on’ by hash collisions leading to false positives i,e a non-existing element may be reported to exist and the goal is to minimise this.

On hash functions

Continue on … clevertap blog

 

I work with these amazing people

And Some of our work