Keeping up with the literature.
WHY IT MATTERS
The field moves quickly, and much useful work appears only in papers.
WHAT ORDER TO READ IN
Title and abstract Figures and tables Conclusion Introduction Method, if still interested Results in detail
WHY THAT ORDER
Most papers are not relevant, and this establishes that quickly.
WHAT TO ASK
What problem is claimed to be solved What was compared against What data was used What was held constant What the paper does not claim
WHAT TO BE SCEPTICAL OF
Comparisons against weak baselines Improvements within the range of random variation Results on one dataset Absence of ablation showing which component matters
WHAT AN ABLATION IS
Removing parts to show each contributes.
WHY ITS ABSENCE MATTERS
Without it, the claimed mechanism may not be the cause.
WHAT TO CHECK
Whether code and data are available.
WHAT TO DO WITH A PROMISING RESULT
Reproduce it on your own data before adopting it.
WHAT TO EXPECT
That many results do not transfer.
