author

Jieyu Zhang

PhD
,
University of Washington

Jieyu Zhang (https://jieyuz2.github.io/) is a PhD student at the University of Washington. His research interests include data-centric AI and agentic AI. He has published multiple first-authored papers in NeurIPS, ICLR, ICML, and so on. He is a recipient of the Apple AI/ML Scholarship and OpenAI Superalignment fellowship. His work has been awarded as the best paper at the LLM Agents workshop @ ICLR 2024 and the best paper at ICDM 2020.

The latest from Jieyu

Task Me Anything: innovating multimodal model benchmarks
Blog
Task Me Anything: innovating multimodal model benchmarks

“Task Me Anything” empowers data scientists to generate bespoke benchmarks to assess and choose the right multimodal model for their needs.

Sep 04, 2024 •
Learn more about Task Me Anything: innovating multimodal model benchmarks
Characterizing the Impacts of Semi-supervised Learning for Weak Supervision
Labeling training data is a critical and expensive step in producing high accuracy ML models, whether training from scratch or fine-tuning. To make labeling more efficient, two major approaches are programmatic weak supervision (WS) and semi-supervised learning (SSL). More recent works have either explicitly or implicitly used techniques at their intersection, but in various complex and ad hoc ways. In this work, we define a simple, modular design space to study the use of SSL techniques for WS more systematically. Surprisingly, we find that fairly simple methods from our design space match the performance of more complex state-of-the-art methods, averaging...
Research Paper
Characterizing the Impacts of Semi-supervised Learning for Weak Supervision

Labeling training data is a critical and expensive step in producing high accuracy ML models, whether training from scratch or fine-tuning. To make labeling more efficient, two major approaches are programmatic weak supervision (WS) and semi-supervised learning (SSL). More recent works have either explicitly or implicitly used techniques at their intersection, but in various complex and ad hoc ways. In…

Jan 16, 2024 •

Jeffrey Li, Jieyu Zhang, Ludwig Schmidt, Alex Ratner

Learn more about Characterizing the Impacts of Semi-supervised Learning for Weak Supervision
On the Tradeoff of Intra-/Inter-class Diversity for Supervised Pre-training
Pre-training datasets are critical for building state-of-the-art machine learning models, motivating rigorous study on their impact on downstream tasks. In this work, we study the impact of the trade-off between the intra-class diversity (the number of samples per class) and the inter-class diversity (the number of classes) of a supervised pre-training dataset. Empirically, we found that with the size of the pre-training dataset fixed, the best downstream performance comes with a balance on the intra-/inter-class diversity. To understand the underlying mechanism, we show theoretically that the downstream performance depends monotonically on both types of diversity. Notably, our theory reveals that...
Research Paper
On the Tradeoff of Intra-/Inter-class Diversity for Supervised Pre-training

Pre-training datasets are critical for building state-of-the-art machine learning models, motivating rigorous study on their impact on downstream tasks. In this work, we study the impact of the trade-off between the intra-class diversity (the number of samples per class) and the inter-class diversity (the number of classes) of a supervised pre-training dataset. Empirically, we found that with the size of…

Oct 20, 2023 •

Jieyu Zhang, Bohan Wang, Zhengyu Hu, Pang Wei Koh + 1 more

Learn more about On the Tradeoff of Intra-/Inter-class Diversity for Supervised Pre-training
MaskSearch: Querying Image Masks at Scale
Machine learning tasks over image databases often generate masks that annotate image content (e.g., saliency maps, segmentation maps, depth maps) and enable a variety of applications (e.g., determine whether a model is learning spurious correlations or if an image was maliciously modified to mislead a model). While queries that retrieve examples based on mask properties are valuable to practitioners, existing systems do not support them efficiently. In this paper, we formalize the problem and propose MaskSearch, a system that focuses on accelerating queries over databases of image masks while guaranteeing the query result accuracy. MaskSearch leverages a novel indexing technique...
Research Paper
MaskSearch: Querying Image Masks at Scale

Machine learning tasks over image databases often generate masks that annotate image content (e.g., saliency maps, segmentation maps, depth maps) and enable a variety of applications (e.g., determine whether a model is learning spurious correlations or if an image was maliciously modified to mislead a model). While queries that retrieve examples based on mask properties are valuable to practitioners, existing…

Oct 20, 2023 •

Dong He, Jieyu Zhang, Maureen Daum, Alex Ratner + 1 more

Learn more about MaskSearch: Querying Image Masks at Scale
DataComp: In search of the next generation of multimodal datasets
Multimodal datasets are a critical component in recent breakthroughs such as Stable Diffusion and GPT-4, yet their design does not receive the same research attention as model architectures or training algorithms. To address this shortcoming in the ML ecosystem, we introduce DataComp, a testbed for dataset experiments centered around a new candidate pool of 12.8 billion image-text pairs from Common Crawl. Participants in our benchmark design new filtering techniques or curate new data sources and then evaluate their new dataset by running our standardized CLIP training code and testing the resulting model on 38 downstream test sets. Our benchmark consists...
Research Paper
DataComp: In search of the next generation of multimodal datasets

Multimodal datasets are a critical component in recent breakthroughs such as Stable Diffusion and GPT-4, yet their design does not receive the same research attention as model architectures or training algorithms. To address this shortcoming in the ML ecosystem, we introduce DataComp, a testbed for dataset experiments centered around a new candidate pool of 12.8 billion image-text pairs from Common…

Oct 20, 2023 •

Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase + 30 more

Learn more about DataComp: In search of the next generation of multimodal datasets
Leveraging Instance Features for Label Aggregation in Programmatic Weak Supervision
Programmatic Weak Supervision (PWS) has emerged as a widespread paradigm to synthesize training labels efficiently. The core component of PWS is the label model, which infers true labels by aggregating the outputs of multiple noisy supervision sources abstracted as labeling functions (LFs). Existing statistical label models typically rely only on the outputs of LF, ignoring the instance features when modeling the underlying generative process. In this paper, we attempt to incorporate the instance features into a statistical label model via the proposed FABLE. In particular, it is built on a mixture of Bayesian label models, each corresponding to a global...
Research Paper
Leveraging Instance Features for Label Aggregation in Programmatic Weak Supervision

Programmatic Weak Supervision (PWS) has emerged as a widespread paradigm to synthesize training labels efficiently. The core component of PWS is the label model, which infers true labels by aggregating the outputs of multiple noisy supervision sources abstracted as labeling functions (LFs). Existing statistical label models typically rely only on the outputs of LF, ignoring the instance features when modeling…

Aug 02, 2023 •

Jieyu Zhang, Linxin Song, Alex Ratner

Learn more about Leveraging Instance Features for Label Aggregation in Programmatic Weak Supervision
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
Large language models (LLMs) have been recently leveraged as training data generators for various natural language processing (NLP) tasks. While previous research has explored different approaches to training models using generated data, they generally rely on simple class-conditional prompts, which may limit the diversity of the generated data and inherit systematic biases of LLM. Thus, we investigate training data generation with diversely attributed prompts (e.g., specifying attributes like length and style), which have the potential to yield diverse and attributed generated data. Our investigation focuses on datasets with high cardinality and diverse domains, wherein we demonstrate that attributed prompts outperform...
Research Paper
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias

Large language models (LLMs) have been recently leveraged as training data generators for various natural language processing (NLP) tasks. While previous research has explored different approaches to training models using generated data, they generally rely on simple class-conditional prompts, which may limit the diversity of the generated data and inherit systematic biases of LLM. Thus, we investigate training data generation…

Jun 28, 2023 •

Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng + 4 more

Learn more about Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
Binary Classification with Positive Labeling Sources
To create a large amount of training labels for machine learning models effectively and efficiently, researchers have turned to Weak Supervision (WS), which uses programmatic labeling sources rather than manual annotation. Existing works of WS for binary classification typically assume the presence of labeling sources that are able to assign both positive and negative labels to data in roughly balanced proportions. However, for many tasks of interest where there is a minority positive class, negative examples could be too diverse for developers to generate indicative labeling sources. Thus, in this work, we study the application of WS on binary classification...
Research Paper
Binary Classification with Positive Labeling Sources

To create a large amount of training labels for machine learning models effectively and efficiently, researchers have turned to Weak Supervision (WS), which uses programmatic labeling sources rather than manual annotation. Existing works of WS for binary classification typically assume the presence of labeling sources that are able to assign both positive and negative labels to data in roughly balanced…

Mar 15, 2023 •

Jieyu Zhang, Yujing Wang, Yaming Yang, Yang Luo + 1 more

Learn more about Binary Classification with Positive Labeling Sources
Understanding Programmatic Weak Supervision via Source-aware Influence Function
Programmatic Weak Supervision (PWS) aggregates the source votes of multiple weak supervision sources into probabilistic training labels, which are in turn used to train an end model. With its increasing popularity, it is critical to have some tool for users to understand the influence of each component (e.g., the source vote or training data) in the pipeline and interpret the end model behavior. To achieve this, we build on Influence Function (IF) and propose source-aware IF, which leverages the generation process of the probabilistic labels to decompose the end model's training objective and then calculate the influence associated with each...
Research Paper
Understanding Programmatic Weak Supervision via Source-aware Influence Function

Programmatic Weak Supervision (PWS) aggregates the source votes of multiple weak supervision sources into probabilistic training labels, which are in turn used to train an end model. With its increasing popularity, it is critical to have some tool for users to understand the influence of each component (e.g., the source vote or training data) in the pipeline and interpret the…

Mar 15, 2023 •

Jieyu Zhang, Haonan Wang, Cheng-Yu Hsieh, Alex Ratner

Learn more about Understanding Programmatic Weak Supervision via Source-aware Influence Function
1 2

For models that need to be right. Not just good enough.