Conversations with Shannon Chan

Jul 15, 2023 11 min read

Data Science & Analytics

By Soong Hung Hao and Julian Rocero

Data is the new oil – harnessing its capabilities and pushing its utilities to new boundaries have generated some of this century’s greatest breakthroughs. Despite this, many have yet to fully grasp what data science really is. Currently a Principal Data Scientist at Carousell, Shannon cut his teeth in Silicon Valley, where he fell in love with data science after achieving mastery in Excel VBA. Armed with his technical prowess, he then kickstarted the data science team at Singapore’s home-grown unicorn start-up, where he continues to work with the best and brightest daily. In this interview, he shares candid advice on the expectations of tech-based roles, tips on pivoting one’s career into the tech space, and the reality behind the world of data science.

As a Principal Data Scientist at Carousell, could you introduce yourself and describe your typical work day?

I have been with Carousell for over seven years, and am currently a Principal Data Scientist. In general, my role involves solving business problems through data-driven approaches, which include methods drawn from machine learning and artificial intelligence. For instance, I have overseen the development of features on Carousell which require an understanding of unstructured data, in the form of images or text.

As a senior member of the data science team, I also monitor a broad range of projects that my team is working on at any moment. I usually start each day with meetings that keep me updated on their progress and allow me to unblock problems they may be facing. Some of these obstacles could be technical, such as writing a piece of code that is suitable for deployment. On the other hand, some issues are more related to analysis – perhaps the results of a research effort are unfavourable, and the team needs guidance on the next steps.

Apart from this, I am also an individual contributor and a technical lead. I am expected to provide technical direction with our projects, typically dealing with search, image understanding and advertising domains. I also frequently review code written by my fellow data scientists for feasibility and accuracy.

In the afternoon, I am often engaging with product managers and engineering leads to discuss the implementation of proposed new features, as well as the team’s priorities and deliverable timelines. Finally, the remainder of the day is usually spent reading reports of new app features that are in the pipeline, and preparing such reports for projects that I am leading.

There are two distinct tracks of progression within our team, being the technical and managerial roles. The former focuses on up-levelling and maintaining standards of code within the team. They are often in charge of bringing innovations to life, focusing more on the implementation details of these projects. On the other hand, managers act as the layer where the data science team interfaces with the rest of the company, and serve to gather the right manpower and resources to fulfil the company’s goals. Managers are also in charge of people issues within the team, such as by helping each member chart their career progression and growth. I fall under the technical track, which I had decided early on when I was a junior data scientist.

What qualities should one possess to be successful in your role?

Having a robust set of technical skills is definitely crucial. My role is ultimately an engineering position, so you should be familiar with programming. I often also find myself wrangling with big amounts of data, so these capabilities are generally the core necessities.

On top of this, being able to communicate well is also essential. Many people do not fully understand our responsibilities and capabilities as data scientists; effectively and convincingly conveying our ideas to other stakeholders in the company is key to my role. Data science is also an extremely dynamic field, with new developments every day. Being attuned to such changes by adopting a mindset of continuous learning is very helpful. Another quality I wish to highlight is perseverance – data science is a highly experimental field which inherently poses the risk of failure. It requires us to develop robust evaluation methodologies to objectively test our hypotheses. Many believe that the world of computing is black and white, whereby computers and models developed for a certain purpose will always succeed. In reality, not all ideas or models will be successful, and being able to bounce back from such failures is imperative for your growth as a data scientist.

As a leader, I must also have a vision of how data science can serve the organisation in terms of delivering business value. I do this by identifying applications of what we do in the app, and translating them into a delightful experience for our end users.

Above all, passion is key to a sustainable career in this field. If you have a genuine interest in the industry, you will be naturally inquisitive about new knowledge and developments in the field. As they say, choose a job you love, and you will never have to work a day in your life.

What are some common misconceptions you have encountered about data scientists?

Data science is a very diverse field that has witnessed explosive growth within the past decade or so. I was recently invited to speak at a panel with other data scientists, and we were asked to share some challenges that each of us faces on a daily basis. Despite sharing similar job titles, I discovered that each of us has vastly different responsibilities in our respective roles. I suppose many incorrectly assume that the title of a ‘data scientist’ means that we all perform identical functions within our organisations.

To clarify, data scientists can be roughly grouped based on who consumes our output – generally speaking, our work is either for machines or humans. I fall under the former category, and my job involves choosing a set of data, converting it into a suitable model, and executing a product feature using the said model. For example, the listing recommendations that a user sees on the app are dependent on their browsing history, which is a feature built into our algorithm. My role hence more closely resembles that of a machine learning engineer, where I ultimately aim to satisfy our end users.

On the other hand, the latter specialises in extracting key findings from data, and reporting these learnings to their managers and business leaders. This form of data science would be more closely linked with data analytics and statistics. However, it is often overlooked for the former, since many believe that data science is only related to learning. In reality, data science simply refers to digesting a piece of data and translating it into an actionable solution that thereby improves a function.

I am often asked what I do at Carousell, and I usually use this example. When sellers list on our site, they are first required to upload a photo of their item. Our model allows us to understand this never-before-seen image, and match it to the appropriate category it belongs in. After a suitable category has been selected, our app is then able to suggest a title and price for the item by scanning through similar listings. Digesting unstructured data in the form of images and text is one of the many data science touchpoints that our team is responsible for.

How would you describe the culture within your team at Carousell? What does it take for one to succeed in such an environment?

Our team has a strong culture of knowledge sharing, which I believe is important for any data science team. This is because developments within our field occur very quickly, and no single individual can know everything. Our team also has rather diverse interests, so sharing our learnings allows us to benefit from each other’s skill sets and level up as a team. We hold knowledge-sharing sessions once a week, to run through recent developments or personal projects.

Apart from this, we are also a very cooperative team. While each data scientist may be attached to a particular function, we are always willing to chip in with ideas that can help propel our teammates towards their goals.

What are some of the biggest challenges you have encountered in your role, and how do you overcome them?

Particularly in my role, it is important to be able to convey my proposed applications of data science to the business stakeholders I work with. Even if I can successfully build a perfect model, I still need to convince product teams that the model creates a meaningful impact and a delightful experience for our end users.

I was at Hackwagon Academy for just over a year, before the pandemic happened. There was a huge interest in data science at the time, and I was keen to share my skills. After all, I strongly believe that up-skilling should be a community effort, and have always been passionate about sharing my knowledge. I am incredibly proud of the work that the Hackwagon team accomplished. It truly helped the impact of data science in Singapore to grow. Each of our sessions hosted about 50 students, and the aim was to simply familiarise them with the fundamentals of data science through engaging practices. They were even able to learn about some basic applications of data science, such as predicting the prices of properties around Singapore and summarising news articles. Overall, it was a fantastic experience that was greatly fulfilling for everyone involved.

We note that you pursued a degree in Accountancy and Finance at NUS; how was your experience in higher education, and how did it play a part in your career?

When I finished my A-Levels, I had an aversion to coding, math, engineering, and the like. In hindsight, it was because of these naïve assumptions and lack of direction that I decided on reading accounting. Despite this, I thoroughly enjoyed my time in university – accounting remains a highly rigorous major, and it built my strong work ethic and collaborative spirit as a team lead. I completed about six internships at accounting firms and finance institutions, and eventually realised that I was not cut out for a career in finance.

While my coursemates were applying for jobs at the Big 4 of accounting, I elected to take a break at the NUS Overseas College for a year, where I chose not to clear any modules and instead delay my graduation. During this time, I picked up part-time jobs in Silicon Valley, and immersed myself in the world of technopreneurship. I honed my skills in coding, then eventually graduated from NUS and was offered a role at Carousell as their first data scientist.

Apart from your time in Silicon Valley, what inspired you to pivot into the tech industry instead of pursuing a career in finance?

I became very well-versed with Microsoft Excel while studying Accountancy, to the point where I did not need to use a mouse at all and everything was done through keyboard shortcuts in Excel. I discovered that my passion lies in building highly automated Excel spreadsheets in VBA, the programming language behind Excel, instead of the valuations and projections that my spreadsheets were tabulating. I enjoyed the logic behind the data, and had a strong propensity in coding to accompany it. Although data science was still in its early stages back then, I pursued this passion relentlessly, and became one of the first members of the industry.

I recommend that students attend as many internships as possible to determine if the working environment in their respective industries is a match to their personality.

What was your greatest takeaway from your time in Silicon Valley as part of the NUS Overseas College program?

My time in Silicon Valley was a very liberating experience, and I learned never to limit myself to my role. It was my first exposure to a very international working environment; despite working in a small start-up, we had a highly multinational culture with highly varied perspectives. Even though my role was a junior data scientist, I found myself working on many different projects that managers wanted my help on. This dynamism was what allowed me to learn so much from that experience, and instilled within me a sense of initiative.

What advice would you give to university students who may be feeling jaded or hesitant to continue in a career path typical for their degree after graduation?

I participated in many internships during my university years, including at banks, innovation labs, small accounting firms, and large auditing companies. Besides being a way to earn some cash, they allowed me to gain meaningful, accurate insights into what life is really like at these organisations. It was only through these experiences that I became very determined to break out of the finance world, because I did not see myself thriving in a typical finance role.

Data science can be an intimidating field for young people to break into, and many may feel that if they have not built a background from a young age, they could already be starting behind the eight ball. What advice do you have on this matter?

If you want to secure a technical role in the tech sector, you certainly need the technical skills to back it up. However, there are still many roles within the tech industry that do not require hard technical skills, but rather some other types of domain knowledge, such as Product Design, User Experience, Program management, Marketing etc. Being in such roles can provide you with the opportunity to learn about what other people in the company (eg. Software engineers or data scientists) do on a daily basis. With this understanding, I believe you will be in a much better position to hone the necessary skills to land yourself a technical position, and pivot to a technical job.

Another tip is to read up on job descriptions of your dream job to find out what it is that potential employers require of candidates. Starting with the end in mind allows you to work backwards, and build your skills accordingly to match their expectations. However, keep in mind that job descriptions are often simply a company’s wishlist for the ideal candidate. So while it may be a good indicator of desirable skills for the job you want, don’t be disheartened if you aren’t equipped with all the skills.

The data science industry, broadly speaking, might also be a field where jobs are notoriously difficult to nail down due to the glut of talent both locally and abroad. Do you have any advice for graduates looking to score their first job?

The fact is that there is also great demand for qualified data scientists, emphasis on qualified. My advice is not to worry about the competition, because the question is whether you, yourself, are able to deliver. You should aim to cultivate a strong portfolio of projects that showcase your capabilities in data science. While internships or contributions to open-sourced projects are good, you can also work on personal projects as there are many data sources online that you can build models upon, thereby gaining expertise and breadth within the data science field. It’s always best to start a personal project from solving a need that you observe personally or from people around you.

What have been your biggest highlights from 2022?

Professionally, I introduced a new way to understand search queries within Carousell. Structured search is a new feature that can parse search queries for understanding various models, brands, colours, and attributes of listed items then help users search for items much more easily on the app. Apart from this, I also rewrote the code base for our machine learning pipelines to keep it relevant to recent developments in technology.

On a more personal front, astrophotography is one of my biggest hobbies, and I took the jump to purchase a telescope and photograph some of the planets. I was very proud to be able to capture and develop a nice photo of the planets Jupiter and Saturn that matched perfectly with the simulated versions. These are some of the biggest highlights of 2022 that I truly take pride in.

In hindsight, is there any advice you wish you had been given prior to entering this field?

You will meet many intelligent people and inevitably feel insufficient or insecure. Getting your foot in the door is difficult, but even after you break into the industry, you will find that the world of data science is saturated with incredibly talented people. On top of the tight competition, there will be prolonged periods of failure where you are unable to produce anything of value, simply because you are out of luck. During these times, just keep your head down and focus on what you do best; keep persevering and take things one at a time. Eventually, when you look up, you will realise you are not that far from being a capable data scientist after all!