Swidden agriculture is by far the dominant land use system in the mountainous regions of Southeast Asia (SEA). It provides various valuable subsistence products to local farmers, mostly the poor ethnic minority groups. Controversially, it is also closely connected with a number of environmental issues. With the strengthening regional economic cooperation in SEA, swidden agriculture has experienced drastic transformations into other diverse market-oriented land use types since the 1990s. However, there is very limited information on the basic geographical and demographic data of swidden agriculture and the socio-economic and biophysical effects of the transformations. International programs, such as the Reducing Emissions from Deforestation and forest Degradation (REDD), underscore the importance of monitoring and evaluating swidden agriculture and its transition to reduce carbon emission due to deforestation and forest degradation. In this context, along with the accessibility of Landsat historical imagery, remote sensing based techniques will offer an effective way to detect and monitor the locations and extent of swidden agriculture. Many approaches for investigating fire occurrence and burned area can be introduced for swidden agriculture mapping due to the common feature of fire relatedness. In this review paper, four broad approaches involving spectral signatures, phenological characteristics, statistical theory and landscape ecology were summarized for swidden agriculture delineation. Five research priorities about swidden agriculture involving remote sensing techniques, spatial pattern, change, drivers and impacts were proposed accordingly. To our knowledge, a synthesis review on the remote sensing and outlook on swidden agriculture has not been reported yet. This review paper aims to give a comprehensive overview of swidden agriculture studies in the domains of debated definition, trends, remote sensing methods and outlook research in SEA undertaken in the past two decades.