THIS IS AN UNFINISHED ARTICLE
Getting a job, no matter if internship or full-time, in Silicon Valley is a dream of many. And it is the many that can make it difficult to get a job. Afterall, the job market is a supply/demand market. The less demand a job offer has, the easier it can be to get the job.
There are a few dedicated startup job platforms:
| http://venturefizz.com/jobs For boston and NY | |
| https://www.themuse.com/jobs?company=The%20Muse | |
| AsiaWired.com – Looking for startups in Asia? This may be the solution for you. | |
| CoNotes.com – Focusing on nothing but jobs at startups, CoNotes has been around since 2007. | |
| Dice.com – Browse jobs by city or pull up the category that applies to your skill set. | |
| Jobs.Mashable.com – Our very own marketplace features categories for listing jobs and looking for them also. | |
| NeoHire.com – Lets you look up jobs by category, add them to your basket as you find ones that interest you and then apply to all of the ones you’ve saved. | |
| nPost.com – Besides offering numerous job listings at startups, they have 225+ interviews with people from some of the companies explaining what they are about and what they are looking for in an employee. | |
| VentureLoop.com – Provides internship listings for students at certain schools and has job listings you can search by country or occupation. |
While it is more convenient to simply go to Angel.co or StartUpHire and search through the database of job offers, there are many startups that don’t put their job openings on these platforms.
I will outline in the following an approach to find job applications that are likely to have less applicants – that are less advertised on job platforms.
We will circumvent the drawbacks of applying to jobs that are publicly advertised (e.g. on angel.co) by checking each Silicon Valley startup’s own job listings on their websites. But instead of doing this manually, we will automate most of the steps with the scraping tool KimonoLabs.
| STEPS | Automatization | Time needed |
| Step 1) Click through startup databases and find a company url | automated | |
| Step 2) Load the website, scan the website to find the page with job offers | automated | |
| Step 3) Analyse relevancy of job postings | automated | |
| Step 4) If there are no job offers, search their email address for cold applications. | automated | For step 1-4: on average 30 seconds. For 1.000 startups that is 5.5 hrs |
I am in my second year of my undergraduate studies and haven’t narrowed down my passion/career direction. I like many things, from product management, to marketing, to business development. Alternatively, in case you have a clear role you are looking for,
Step 1: Acquiring a dataset of companies
First, we will need to get a list of startups and their websites. We will use the databases of CrunchBase and Angel.co for this purpose.
To get a large data set of URLs of startups, we scrape a variety of sources.
Startup list curators: https://angel.co/companies?locations[]=Silicon+Valley&company_types[]=Startup
and https://www.funderbeam.com/startups?page=99999
We are adding an accelerator: 500startups: http://www.500.co/startup
And a VC firm: http://www.accel.com/#companies/2wire-acquired-by-pace
And a job search platform: https://www.google.co.uk/search?hl=en&q=site:http://www.ventureloop.com/ventureloop/companyprofile.php&gws_rd=cr&ei=1lY6Vfu4CorSaLL8gMgC&start=0&num=100
Note: The selection of sources is crucial for the usefulness of this whole process. Select the search parameters on each data source wisely (e.g. limiting to startups located in Sunnyvale). Of course, this is not just limited to startups.
To scrape all these sources and export the URLs of our startups, we will use KimonoLabs. (a word on Kimono)
We now export all URLs and import them in Excel to combine them and get rid of duplicates and false websites. (Strip away “http://”, “www.” and “https:/” first and then get rid off obvious false positives such as *oracle.com/* and all URLs that link to sub-pages */*/. This is important and lead me to remove another 472 duplicates).
This leaves me with a final XXX startups that I have uploaded here – feel free to add new startups to the list!
Step 2: Scrape websites for relevant job offers
Key to this step Google’s search engine and an appropriate search query.
We will use the following search query to search each website for marketing internships, but experiment yourself with a few example companies to find the right search query:
site:*company_url*+marketing intern OR internship
Kimono scraping results:
Limitations:
It will not show job offers if the startup is exclusively using an outside provider to list the job offers (e.g. silkroad.com). Since we are looking for early stage (seed and series A, maybe B) startups, most will post their job offers on their website. Hence, this “limitation” may actually work in our favour. We won’t see job offers of any established big players.
Alternative A: Search for email for cold applications
site:*company_url*+jobs@*company_url* OR careers@*company_url* OR founders@*company_url*
Alternative B:
Bottom-line
I took away two things from this weekend-project:
- Web scraping (i.e. the automation of intelligence gathering) is powerful for all sorts of decision making. Kimono was helpful.
- Google, and search filters on Google and the individual sources is powerful and essential to know how to use.
The bigger picture: using scraping to automate day-to-day decision making
I plan on writing a blog post about jobs that “allow remote”, “location independent”, “worldwide”.
Also, set up Google Alerts https://www.google.com/alerts