AI 검색 결과

해당 글 38건

Self-Supervised Representation Learning with Meta Comprehensive Regularization

AAAI 2024 Accepted PaperLink: https://arxiv.org/pdf/2403.01549  Summaryaugmentation을 하면 downstream task에 필요한 정보가 손실이 될 수도 있다.그렇기 때문에 comprehensive representation을 학습할 수 있도록 해야한다.최대한 많은 semantic information을 담아내는 representation을 얻기 위해서 information theory 관점상 Entropy를 최대화 하는 방향으로 가야한다.두 loss의 목적은 비슷: 표현을 풍부하게 만들어 downstream task 성능 향상 하지만 적용 위치가 다름$L_{\text{comp}}$: backbone에서 추출되는 전체 표현의 엔트로피를 높임..
AI 2025. 3. 5. 19:00

PDiscoFormer: Relaxing Part Discovery Constraints with Vision Transformers

ECCV2024 Accepted PaperLink: https://arxiv.org/pdf/2407.04538 SummaryPDiscoFormer는 비전 트랜스포머를 기반으로 한 새로운 비지도 부분 발견 접근 방식으로, 기존의 엄격한 기하학적 가정들을 완화하는 데 초점을 맞추고 있습니다.본 논문의 주요 목표는 fine-grained image classification task에 도움이 되는 구분 가능한 부분들을 자동으로 식별하는 것입니다.Method Obtain $z$ via drop cls token and register tokens$\mathbf{z} = h_{\theta}(\mathbf{x}) \in \mathbb{R}^{D \times H \times W}$Attention Maps: $\ma..
AI 2025. 3. 5. 19:00

Lost and Found: How Self-Supervised Learning Helps GPS Coordinates Find Their Way

ACML Accepted Paper.Link: https://proceedings.mlr.press/v222/bougie24a/bougie24a.pdf SummaryTo capture the rich underlying semantics of GPS coordinatesauxiliary tasks including geo **predictionhigh-level reconstructionintermediate clustering.each GPS coordinate is projected onto a map image centered on the input coordinate.Then, a student and teacher networks receive two different augmented vers..
AI 2025. 3. 5. 19:00

Contrastive Knowledge Distillation from A Sample-wise Perspective

Link: https://arxiv.org/pdf/2404.14109Summary$\mathcal{L}_{\text{intra}} = \frac{1}{n} \sum{i=0}^{n} d\left( t_i, s_i \right)$위 수식에서 $t_i$와 $s_i$가 가깝도록 학습을 하게 되는데, 이게 $d\left( t_i, s_i \right) Loss가 0이더라도 student가 teacher의 내부 표현의 구조나 결정 경계를 배움에는 한계가 있다.Teacher’s raw score before softmax: [0.4, 0.4], [0.6, 0.6] → [0.5, 0.5], [0.5, 0.5]Student’s raw score before softmax: [1.4, 1.4], [5.6, 5.6] → [..
AI 2025. 2. 26. 19:00

From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labels

ICCV'23 Accepted Paper.Link:  https://arxiv.org/pdf/2303.13005 MethodNormalized Knowledge Distillation (NKD) loss function$L_{nkd} = - T_t \log(S_t) - \gamma \cdot \sum_{\substack{i=1 \\ i \neq t}}^{C} \mathcal{N}(T_i) \log \left( \mathcal{N}(S_i) \right)$제안하는 logit-based KD 방법론인 NKD는 SoTA로 보입니다만, 코드 구현이나 논리는 DKD와 비슷한 측면이 있습니다.저자도 의식해서 해당 부분을 설명하고는 있지만 코드는 gt label index가 들어갔냐 아니냐 정도 차이입니다.NKD l..
AI 2025. 2. 25. 19:00

Open-World Panoptic Segmentation

Link: https://arxiv.org/abs/2412.12740  Open-World Panoptic SegmentationPerception is a key building block of autonomously acting vision systems such as autonomous vehicles. It is crucial that these systems are able to understand their surroundings in order to operate safely and robustly. Additionally, autonomous systems deploarxiv.org 자율주행 및 실제 환경의 복잡성자율주행 차량/로봇 등 실제 시스템은 다양한 환경에서 센서 데이터를 해석해야 ..
AI 2025. 2. 17. 19:00

Relation DETR: Exploring Explicit Position Relation Prior for Object Detection

ECCV 2024 Accepted Paper.Link: https://arxiv.org/pdf/2407.11699  Summarysuggesting that it arises from the self-attention that introduces no structural bias over inputsintroduces an encoder to construct position relation embeddings for progressive attention refinementfurther extends the traditional streaming pipeline of DETR into a contrastive relation pipeline to address the conflicts between n..
AI 2024. 12. 9. 09:45

FS-DETR: Few-Shot Detection Transformer with prompting and without re-training

ICCV 2023 Accepted PaperLink: https://arxiv.org/abs/2210.04845 SummaryFSOD system must fulfil the following desiderata:it must be used as is, without requiring any fine-tuning at test timeit must be able to process an arbitrary number of novel objects concurrentlywhile supporting an arbitrary number of examples from each classit must achieve accuracy comparable to a closed systemfew-shot detecti..
AI 2024. 11. 17. 00:49