Business Analytics Related Jobs and the H1-B Visa
A data cleaning and visualization project
Authors: Anika Abrahamson, Jasmine Yao, Wendy Wang
Why We’re Interested
As Duke University Fuqua School of Business Master’s of Quantitative Management in Business Analytics students, we wanted to gain insights on jobs related to our degree and people who’ve received the H1-B visas in the last 6 years. Many MQM students with non-permanent work authorization wish to work in the U.S. after graduation and we were hoping to find information on how the companies, salaries, and locations where H1-B visas were granted have changed from 2015 through 2020. In this blog post, we describe our process of finding and cleaning data, then we discuss how we used Tableau to gain insights from this data.
The Data
Our Sources
We obtained information on H1-B visas from the U.S. foreign-labor performance data found here. Price parity data from the Bureau of Economic Analysis at the U.S. Department of Commerce was also obtained so we could more easily compare salary information from different states. This data can be found at the link here.
Clean-up time
As with every data science project, our first step to discoveries was cleaning the data we sourced. This process was the most time consuming but made a huge difference once we imported our data to our data visualization platform, Tableau. Our first roadblock arrived when we found out combining the past 6 years of H1-B data was way over a million rows, and thus couldn’t be made into one sheet in excel. We decided to initially filter out any jobs that aren’t related to positions that MQM students are likely to take. After combining the misspelled iterations of different positions, in the end, there are 52 job subgroups ranging from accounting, marketing analytics, financial analysts, and management analytics to business intelligence, operations specialists, computer programmers, and everything in between. We felt this group specifically related to our fellow Master’s students, while also capturing the range of business analytics positions that would be interesting to others as well.
Once the data was paired down to these 52 job subgroups we were then able to focus on the columns most important to the questions we had, these are
In our final cleaning step, we use the R to make sure all the records are consistent in unit, abbreviation, and format. After finishing the cleanup, we were ready to jump in and try to find answers to our questions.
Why Tableau?
For this project, we realized the best way to gain the insights we were focused on wasn’t through coding algorithms like regressions or using machine learning like a random forest. For these reasons, we knew Tableau, and data visualization was the way to go. Because Tableau allows for the user to create calculations and the ability to move around data in a way similar to SQL it’s the perfect platform to help gain insights through data visualization. We are also able to create dashboards that combine and enhance our visualizations. The final reason we were drawn to Tableau was the ability to easily join our price parity data with the H1-B dataset which allows us to more accurately compare salaries of similar jobs in different states.
Our Findings
Our findings are broken down into four major areas, job-related information, visa type, employer information, and wage information. The following sections of this post will focus on the questions we had within each of these categories and the answers we found through our visualizations.
Job-related Information
The first information we wanted to see was which job has the greatest number of total certified H1-B visas over the last 6 years. As shown below, computer system analysts are granted significantly more visas than any other business analyst/data-related jobs. We believe this could be due to the specialized skill set needed for these jobs and the increased number of these positions over the past 6 years. For Duke MQM students, it’s positive to see so many varieties of analysts in the top 10, meaning no matter what your passion or area of expertise, you do have a shot at a certified H1-B visa in the industry you wish to focus your career.
Up next, we gained insight into the percentage of applications certified each year. The graph below reveals that certified applications have increased each year and are most recently at 96%. Knowing that the Covid-19 pandemic affected most jobs in the U.S. in 2020, we created the line graph below to show application trends. As suspected, the number of applications has significantly dropped from 2019–2020. So although 96% of the applications were certified in 2020, it was still fewer than the number certified in 2019.
Lastly, we created an interactive dashboard showing the median wage, median prevailing wage, and the percent difference between the two for each job subgroup. The line graphs on the top display how the median wage for that job has changed between 2015 and 2020. This dashboard can help demonstrate the trends for each job position. As shown in the image of the dashboard below, Business Intelligence Analysts, compared to Broadcast News Analysts, have a median wage that is much greater than the prevailing wage. This means they are getting paid higher than median salaries for that job. After playing around with our dashboard we noticed many of the analytics related jobs get paid more than the median prevailing wage. To interact with our dashboard, click on the link here.
The H1-B Visa and Its Counterparts
Although many know of the H1-B visa, there are also special agreements the U.S. has with Australia, Chile, and Singapore. Citizens of these countries can apply for not only the H1-B but also their country’s other visa if they so choose. We wanted to examine the acceptance rates between these visas and compare them to the H1-B. Overall, we can see that the H1-B approval rate for the jobs is above 90% for regular H1-B as well as these special cases. It seems that the special agreement doesn’t boost the approval rate, at least not in the jobs that MQM graduates are interested in applying for. Additionally, if we zoom in to look at the individual job position’s approval rate, then it varies. For example, a computer programmer analyst has a 100% chance of being certified while marketing managers has 93.9% of being approved of H1-B sponsor. We found that all the jobs in the job subgroup that we selected have a high approval rate. To interact with the graph, click here.
What About The Employers?
The U.S. foreign labor performance data doesn’t only contain information on the jobs, but it also includes employer information and location. First let’s look into the locations of the companies that sponsor H1-B visas across the contiguous United States, which was plotted based on the postal codes of the worksites. This map, shown below, can give us an overview of which regions are better choices in terms of job finding. The graph below reveals the locations of companies, which are unevenly distributed across the country, gathered more in large cities and the coasts. The red dots indicate the companies that sponsored more than 100 employees between 2015 and 2020, which we regard as top corporate sponsors. These companies are usually located in big cities including Seattle/San Francisco/Los Angeles in the west, Chicago and the Twin Cities in the midwest, New York/Boston/Washington D.C in the east, and Dallas/Orlando in the south. (There’re also some located in North Carolina, which may be particularly intriguing to Duke Students).
After looking at the locations where employers have many H1-B visas certified, we wanted to see which employers sponsor the most H1-B certified visas and how their median salary compares to that of the median prevailing wage. According to the Forbes news, a new H1-B lottery system applied by the Department of Homeland Security will soon be replaced by a salary ranking system, which means that the applicants who are paid with higher than prevailing wage levels will potentially have more chances to be approved. Thus, companies that sponsor more H1-B visas and pay employees higher than prevailing wage salaries are more ideal choices for graduates in terms of employment authorization. Our intuition of the following dashboard is to better visualize the count of sponsors and salary information of the employers so that it can help to find out the ideal employers.
Of note, Infosys Limited has the most certified H1-B visas from 2015–2020, and Merrill Lynch is paying it’s certified H1-B visa employees the highest percentage over the prevailing wage. According to the interaction between the two tables, Ernst & Young (EY) has a high ranking in both dimensions with 3rd place for the number of sponsors and 6th place for the wage. For the lower graph in the dashboard, the red reference line refers to the median value of percent over the prevailing wage among all the H1-B application cases. The histograms with yellow color are the ones above the median percentage, which we regarded as companies with more competitive wages, while the blue ones are below the median. To interact with the dashboard, please click on the link here.
All About The Money
As those hoping to gain certified H1-B status are starting to apply for jobs, we wanted to give them a good idea of which states have the most H1-B visa applications. The map below shows the states with only over 20 thousand H1-B visa applications from 2015–2020. This map shows resemblance to the employer map above in that many of the coastal states, Illinois, and Texas seem to have the highest numbers of H1-B visas.
With the map above in mind, we created an interactive graph to show the median wage of each job subgroup for each of the states shown above. On this graph, we included price parity information so as to adjust the median wage for the cost of living in each of the states of interest. Unsurprisingly, many states that we assume have a higher cost of living, for example, California, have a much lower adjusted paid wage per year than their median paid wage with most job positions. However, there are some states which seem to pay more in comparison to the cost of living than others. North Carolina is a good example of this, when looking at the business intelligence analysts position the median paid wage is $88,000 but when this is adjusted for the cost of living, it actually goes up to $96,000 a year. To interact with our graph, click on the link here.
Final Thoughts
The Impact of Covid-19
From our research, we have realized that the overall numbers in 2020 seem to be quite different than those of the previous 5 years. We believe this is due to the COVID-19 Pandemic. For those trying for certified H1-B visas in business analytics related positions, the overall acceptance rate has been increasing each year, this is good news for our fellow Duke MQM international classmates. However, with the hit of COVID-19, the number of applications, and thus the number of certified visas, dropped significantly in 2020. Before last year, however, it looked like analytics positions were still booming in their numbers of certified visas, and hopefully, with the recovery of the economy and people returning to the workforce, the number of certifications will once again jump back.
We also found the median prevailing wage for many of these positions is much higher than the median paid wage and many jobs’ median paid wages peaked in 2019. Optimistically, we believe that once the vaccine has come out and the economy returns to where it was before the pandemic, the median paid wages will also bounce back.
Locations
After studying our findings, we believe that students hoping to gain H1-B visas should focus their search on the key states and cities, which are hot spots for H1-B visa sponsoring companies, shown in our map above. When taking the price parity into consideration, companies in some states, including North Carolina and Georgia, are actually paying higher median adjusted wages for certain positions than others as shown in our interactive graph.
Jobs
Overall, more managerial or highly skilled positions are certified the most and tend to have a higher median paid wage, especially those related to finance. Our dashboard exhibiting wages for job subgroups shows the trend of median paid wage increasing from 2015 to 2019, but then a decrease in 2020. Once again this could be caused by COVID-19 but it will take a few more years to see if the median paid wage continues to decrease or stabilizes once again.
Employers
In terms of employers, we found that the companies that sponsor most H1-B visas for business analytics-related employees usually also provide competitive wages (i.e. Ernst & Young, PricewaterhouseCoopers, IBM, JP Morgan Chase, etc.). However, there are also some companies that sponsor a large quantity of H1-B visas while providing wages that are lower than the prevailing wage, which we should be aware of when applying for jobs.
Future Changes
Looking forward, it has recently been announced that the process for certifying an H1-B visa will change. According to the proposed rule published by the Department of Homeland Security, the new H1-B cap selection process will generally first select registration based on the highest Occupational Employment Statistics (OES) prevailing wage level, which indicates that applicants with higher wages will be more likely to obtain H1-B visas. Given this new rule, when looking for foreign employees, employers will probably be willing to offer higher wages for positions that require higher-skilled workers rather than using the H1-B program to fill relatively lower-paid vacancies. Since during the years we chose (2015–2020), the lottery system is used for H1-B visa selection, the data may change after the new wage-based policy is implemented. However, the jobs analyzed by our research are among the positions with high wages so that they will be less affected by the change of the rule. Additionally, the wage comparison graphs indicate that the majority of the companies sponsoring most H1-B visas also provide higher than medium wages.
Acknowledgments
From our program at Duke University Fuqua School of Business Master’s of Quantitative Management: Business Analytics, we would like to acknowledge our data visualization professor Dr. Jana Schaich Borg for not only teaching us to use Tableau but also for using other years of this data in previous work and lessons and for getting us interested in the topic.
We would also like to thank Fuqua and Duke for giving us the necessary skills to try a project like this and for giving us the opportunity to enter our work in the MQM winter data competition.
Works Cited
Department of Homeland Security. “Modification of Registration Requirement for Petitioners Seeking To File Cap-Subject H-1B Petitions.” U.S. Department of Homeland Security, 28 Oct. 2020, Accessed 15 Jan 2021https://www.dhs.gov/sites/default/files/publications/20_1028_uscis_h-1b-registration-selection-by-wage-levels-nprm-508.pdf.
“Modification of Registration Requirement for Petitioners Seeking To File Cap-Subject H-1B Petitions.” Federal Register, 8 Jan. 2021, Accessed 23 Dec. 2020.www.federalregister.gov/documents/2021/01/08/2021-00183/modification-of-registration-requirement-for-petitioners-seeking-to-file-cap-subject-h-1b-petitions.
“Performance Data.” U.S. Department of Labor, Accessed 22 Dec. 2020.www.dol.gov/agencies/eta/foreign-labor/performance.
“Regional Price Parities by State and Metro Area.” Regional Price Parities by State and Metro Area | U.S. Bureau of Economic Analysis (BEA), Accessed 23 Dec. 2020.www.bea.gov/data/prices-inflation/regional-price-parities-state-and-metro-area.
Semotiuk, Andy J. “Department Of Homeland Security To Abandon H1B Visa Lottery For Salary Rankings.” Forbes, Forbes Magazine, 28 Oct. 2020, Accessed 2 Jan. 2021 www.forbes.com/sites/andyjsemotiuk/2020/10/28/department-of-homeland-security-to-abandon-h1b-visa-lottery-for-salary-rankings/?sh=1e08e0252c8b.
