About Us | Help Videos | Contact Us | Subscriptions

The Plant Genome - Original Research

SNP Discovery and QTL Mapping of Sclerotinia Basal Stalk Rot Resistance in Sunflower using Genotyping-by-Sequencing


This article in TPG

  1. Vol. 9 No. 3
    unlockOPEN ACCESS
    Received: Mar 30, 2016
    Accepted: July 19, 2016
    Published: September 29, 2016

    * Corresponding author(s): lili.qi@ars.usda.gov
Request Permissions

  1. Zahirul I. Talukdera,
  2. Gerald J. Seilerb,
  3. Qijian Songc,
  4. Guojia Mad and
  5. Lili Qi *b
  1. a Dep. of Plant Sciences, 166 Loftsgard Hall, North Dakota State Univ., Fargo, ND 58102, USA
    b USDA–ARS, Sunflower and Plant Biology Research Unit, Northern Crop Science Laboratory, 1307 18th St. N., Fargo, ND 58102, USA
    c USDA–ARS, Soybean Genomics and Improvement Laboratory, 10300 Baltimore Ave., Beltsville, MD 20705, USA
    d Dep. of Plant Pathology, 306 Walster Hall, North Dakota State Univ., Fargo, ND 58102, USA


Basal stalk rot (BSR), caused by the ascomycete fungus Sclerotinia sclerotiorum (Lib.) de Bary, is a serious disease of sunflower (Helianthus annuus L.) in the cool and humid production areas of the world. Quantitative trait loci (QTL) for BSR resistance were identified in a sunflower recombinant inbred line (RIL) population derived from the cross HA 441 × RHA 439. A genotyping-by-sequencing (GBS) approach was adapted to discover single nucleotide polymorphism (SNP) markers. A genetic linkage map was developed comprised of 1053 SNP markers on 17 linkage groups (LGs) spanning 1401.36 cM. The RILs were tested in five environments (locations and years) for resistance to BSR. Quantitative trait loci were identified in each environment separately and also with integrated data across environments. A total of six QTL were identified in all five environments: one of each on LGs 4, 9, 10, 11, 16, and 17. The most significant QTL, Qbsr-10.1 and Qbsr-17.1, were identified at multiple environments on LGs 10 and 17, explaining 31.6 and 20.2% of the observed phenotypic variance, respectively. The remaining four QTL, Qbsr-4.1, Qbsr-9.1, Qbsr-11.1, and Qbsr-16.1, were detected in only one environment on LGs 4, 9, 11, and 16, respectively. Each of these QTL explains between 6.4 and 10.5% of the observed phenotypic variation in the RIL population. Alleles conferring increased resistance were contributed by both parents. The potential of the Qbsr-10.1 and Qbsr-17.1 in marker-assisted selection (MAS) breeding are discussed.


    BSR, basal stalk rot; CIM, composite interval mapping; DI, disease incidence; DS, disease severity; GBS, genotyping-by-sequencing; LG, linkage group; LOD, logarithm of the odds; MAS, marker-assisted selection; MSR, midstalk rot; NGS, next-generation sequencing; PCR, polymerase chain reaction; QTL, quantitative trait loci; RIL, recombinant inbred line; SNP, single nucleotide polymorphism; SSR, simple sequence repeat

Sclerotinia sclerotiorum (Lib.) de Bary is a necrotrophic fungus with a vast host range of >400 broadleaf species (Boland and Hall, 1994). The fungus causes three distinctly different diseases on sunflower: BSR or wilt, midstalk rot (MSR), and head rot. This fungal pathogen is characterized by its ability to produce long-term survival structures called sclerotia, which individually consist of masses of hyphae surrounded by a hard, black, protective rind. Depending on environmental conditions, sclerotia can germinate myceliogenically, and cause root infection, or carpogenically, producing apothecia then ascospores and infect aboveground parts of host plants (Gulya et al., 1997; Bolton et al., 2006). Unlike other hosts, BSR symptoms start from a root infection resulting from myceliogenic germination of sclerotia. Midstalk rot commonly begins as a leaf infection, while head rot infection begins on capitula. Both MSR and head rot symptoms are incited by airborne ascospores released from carpogenic germination of sclerotia. Midstalk rot is not as common in the United States as BSR and head rot. Meanwhile, BSR and head rot are serious problems in sunflower-growing areas of the humid temperate as well as tropical and subtropical regions of the world (Gulya et al., 1997). As the mode of infection for these two important sunflower diseases varies, the underlying genetics of resistance for the two diseases also appears to be different, effectively doubling the effort needed to combat the pathogens (Talukder et al., 2014b). A literature review revealed that most of the research on resistance to S. sclerotiorum in sunflower was conducted on aboveground parts of the host, for example, head rot or MSR. Head rot resistance QTL have been described in a number of studies. Gentzbittel et al. (1998) reported a major head rot resistance QTL on LG1, explaining up to 50% of the phenotypic variability. Mestries et al. (1998) identified two head rot resistance QTL and together explained 38% of the phenotypic variation in the population. In a 2-yr study, Bert et al. (2002) identified 10 QTL associated with head rot resistance of which three were on LG7, two on LG8, and one of each on LGs 3, 5, 6, 10, and 13, explaining between 9 and 20% of the phenotypic variability. In a later study, Bert et al. (2004) identified five genomic regions associated with head rot resistance: two on LG9 and one of each on LGs 6, 7, and 17, explaining between 2.5 to 19.5% of variation for percentage of attack or latency index in the population. Rönicke et al. (2005) reported two QTL for head rot resistance: one of each on LGs 1 and 10, explaining 17.1 and 10.6% of the total phenotypic variance, respectively. Yue et al. (2008) studied head rot resistance in F2:3 and F2:4 families derived from a cross between HA 441 and RHA 439 and identified nine disease incidence (DI) and seven disease severity (DS) QTL. Two of the DI QTL were identified on LG9 and one of each on LGs 2, 3, 7, 8, 10, 12, and 17, explaining between 9.6 and 26.4% of phenotypic variance. The DS QTL were located on LGs 2, 4, 8, 9, 10, 12, and 13, explaining between 8.4 and 34.5% of phenotypic variance. Zubrzycki et al. (2012) identified a total of 20 head rot QTL for DI, DS, and the area under the disease progress curve on LGs 10, 13, and 14.

Basal stalk rot resistance is genetically complex and conditioned by multiple genes, each having a small effect (Davar et al., 2010; Amouzadeh et al., 2013; Talukder et al., 2014c). Little is known about the QTL for resistance to Sclerotinia BSR in sunflower. Published reports on QTL mapping studies for BSR resistance in sunflower thus far have been limited to one biparental mapping population derived from a cross between PAC2 and RHA266 (Davar et al., 2010; Amouzadeh et al., 2013). In both of those studies, RILs were grown in growth chambers under controlled conditions and inoculated with mycelia plugs placed against the basal stem of the 4-wk-old sunflower plants. Using a moderately aggressive fungal isolate, SSU107, Davar et al. (2010) identified seven QTL for percentage necrotic area localized on LGs 1, 2, 4, 6, 8, 14, and 17 of the sunflower genome. In a later study using a different moderately aggressive fungal isolate (SSKH41), Amouzadeh et al. (2013) identified five QTL on LGs 1, 3, 8, 10, and 17. The phenotypic variations explained by each of the QTL on both the studies were small and ranged between 0.5 to 8%.

No major gene, thus far, has been identified conferring complete resistance against S. sclerotiorum disease complexes in cultivated sunflower. However, significant differences in BSR susceptibility have been reported in inoculated field screening trials involving diverse germplasms (Talukder et al., 2014b). Growing resistant hybrids is the most promising strategy for minimizing the losses attributed to BSR in sunflower. Conventional breeding has been the general method used in developing resistant cultivars against this disease. Although, modern genomics tools have been used successfully in many breeding programs to dissect complex disease traits in other crops, sunflower lags behind in terms of genetic and genomic resources required to address such problems. However, recent advances in sunflower reference genome sequencing (Kane et al., 2011; Grassa et al., 2015) has enabled the discovery of large number of SNP markers by resequencing of other sunflower lines using next-generation sequencing (NGS) technology. Linkage-analysis-based QTL mapping has successfully demonstrated its utility in dissecting the complex traits in several important crops (for review, see Bernardo, 2008). Selection based on informative markers tightly linked to BSR resistance would enhance breeding efficiency and lead to the rapid development of resistant sunflower hybrids.

Our objectives were (i) to develop a genetic map of sunflower using SNP markers generated by GBS approach and (ii) to identify QTL associated with BSR resistance in the cultivated sunflower. We analyzed the potential of the BSR resistance QTL in a practical MAS breeding program and converted the tightly linked flanking SNP markers into polymerase chain reaction (PCR)-based allele-specific markers. Our long-term objective is to introgress resistance QTL into elite sunflower lines using MAS and to develop sunflower hybrids with increased resistance to BSR.

Materials And Methods

Plant Materials and Experimental Design

A sunflower population of 106 F7 RILs was developed by single-seed descent from a cross between the maintainer inbred line HA 441 (PI 639164) and the restorer inbred line RHA 439 (PI 639162). Both of these lines were selected and released in 2003 by USDA–ARS, Sunflower Research Unit and North Dakota Agricultural Experiment Station at Fargo, ND, for their tolerance to Sclerotinia head rot (Miller et al., 2005; Miller and Gulya, 2006). Greenhouse and field screening trials at multiple locations in North Dakota, South Dakota, and Minnesota from 2008 to 2014 revealed that HA 441 and RHA 439 were moderately to highly tolerant to BSR (data not shown).

All 106 RILs, along with the parents, were evaluated for BSR resistance at five environments (locations and years) in North Dakota and Minnesota. Field trials were conducted at Carrington, ND, in 2012 and 2014; at Crookston, MN, in 2012 and 2013; and at Grandin, ND, in 2014. All field screening trials were conducted using a randomized complete block design. The 2012 and 2013 field trials were conducted with two replications, while the 2014 trials had four replications. Each plot consisted of a single 6-m row with a spacing of 75 cm between rows and thinned to 25 plants per row. Commercial oilseed hybrid ‘Croplan 305’ was used as the resistant check, while ‘Cargill 270’ and HA 89 were the susceptible checks. Fields were artificially inoculated with S. sclerotiorum isolate, NEB-274 at the V-6 growth stage (Schneiter and Miller, 1981) following the method proposed by Gulya et al. (2008) by depositing 90 g of S. sclerotiorum mycelia grown on proso millet in furrows of 5 to 7 cm depth on one side next to the row.

Phenotypic Evaluation and Statistical Analysis

The typical BSR symptom develops near the base of the sunflower stalk at the soil line with characteristic tan to manila colored lesion girdling the stalk with occasionally visible white mycelium. Plots were evaluated twice for DI with first evaluation at 7 to 9 wk after inoculation and the second evaluation 2 wk later. Disease incidence was expressed as the percentage of plants showing a BSR lesion. An analysis of variance (ANOVA) of DI of the RIL population was performed across all five environments using PROC MIXED in SAS version 9.3 (SAS Institute, 2011). All factors were treated as random effects, except environment, using the model: yijk = μ + li + b(l)ij + gk + glik + eijk, where y is the DI of the kth genotype tested in the jth replication of the ith environment, μ is the overall mean, li is the effect of the ith environment, b(l)ij is the effect of the jth replication nested in the ith environment, gk is the genetic effect of the kth genotype, glik is the interaction effect of the kth genotype and ith environment, and eijk is the random experimental error. The LSD0.05 (least significant difference) was calculated to compare DI means. Variance components were estimated based on the same statistical model and used to estimate the broad-sense heritability (H2) on an entry mean basis following Nyquist (1991): , where is the genotypic variance, is the genotype × environment variance, is the error variance, r is the number of replications, and l is the number of environments. To determine if the BSR evaluation results were correlated across environments, the Spearman’s rank correlations among trials were performed using the statistical package R version 3.2.3 (R Development Core Team, 2015).

Genotyping-by-Sequencing and Single Nucleotide Polymorphism Calling

Genotyping-by-sequencing using the NGS technology was used for simultaneous discovery and genotyping of SNP markers for the HA 441 × RHA 439 RIL population. Total genomic DNA was extracted from 50 mg of leaf tissue from four young seedlings per RIL and from the two parental lines using Qiagen DNeasy 96 plant kit with a modified protocol described by Horne et al. (2004). The quantity and the quality of DNA were determined with a NanoDrop 2000 Spectrophotometer (Thermo Fisher Scientific). Approximately 2.5 μg of genomic DNA from each of the 106 RILs and two parental lines were sent to the Biotechnology Resource Center Genomic Diversity Facility at Cornell University for GBS as described by Elshire et al. (2011) and at http://www.biotech.cornell.edu/brc/genomic-diversity-facility/services. Briefly, GBS libraries were constructed in 96-plex by digesting genomic DNA with the restriction enzyme EcoT221, a six-base cutter (ATGCAT), followed by ligation of a barcode adaptor and a common Illumina sequencing adaptor to the fragmented DNA. The PCR amplification was performed to create the GBS libraries. The resulting libraries were run through a single-end Illumina HiSeq2000 flow cell for sequencing. One RIL that produced <10% of the mean reads per sample was discarded. Raw machine reads produced an average of 1,695,485 reads for the two parents and an average of 2,358,314 reads for the 105 RILs. TASSEL-GBS discovery pipeline version 3.0.166 (http://www.maizegenetics.net/#!tassel/c17q9) (Bradbury et al., 2007) was used to identify good quality sequence reads with barcodes (tags) from the FASTQ files generated during Illumina sequencing. These sequence tags were aligned using the Burrow–Wheelers Alignment tool version 0.7.8-r455 (Li and Durbin, 2009) to the draft sunflower reference genome HA412.v1.0. (http://sunflowergenome.org/early_access/repository/main/pseudomolecules/) and converted into a TagsOnPhysicalMap file for SNP calling using the TASSEL-GBS quantitative SNP caller. Identical SNPs were merged using the MergeDuplicateSNPs, and the SNPs monomorphic to the two parents were removed, which gave a total of 1595 SNP markers. An additional 359 SNPs were excluded because they had >20% missing values, leaving a total of 1236 SNP markers for linkage analysis. The SNPs were named with a prefix of S1 to S17 based on the draft sunflower genome assembly that corresponds to the 17 sunflower LGs followed by a number representing the physical position of the SNP on the genome. However, there were some SNPs with a prefix of S18, which were discovered in scaffolds yet to be assigned a physical position in the genome. The 400 base pair nucleotide sequences flanking the SNP position were retrieved from the draft sunflower reference genome (Supplemental Table S1).

Linkage Mapping

Linkage analysis was performed using JoinMap 4.1 (Stam, 1993; Van Ooijen, 2006). The Chi-square test was used to assess goodness of fit to the expected 1:1 segregation ratio of the markers using the locus genotype frequencies feature of the JoinMap. Markers were assigned to linkage groups applying the independence logarithm of the odds (LOD) parameter with LOD threshold values ranging from 2.0 to 10.0. We used the similarity of loci option of JoinMap to identify perfectly identical markers (similarity value = 1.000), which are supposed to be mapped at exactly the same position on the linkage group. To reduce the load of calculation effort, only one representative of the similar-loci marker was kept for the linkage mapping analysis. Linkage analysis and marker order were performed using the maximum likelihood mapping algorithm. Recombination fractions were converted to map distances in centimorgans (cM) using the Kosambi mapping function (Kosambi, 1943). Markers were assigned to one of 17 LGs, which correspond to the 17 sunflower chromosomes. The excluded similar-loci markers, however, were included in the final map.

Quantitative Trait Loci Mapping

Phenotypic trait distributions in all environments were assessed for normality with a Shapiro–Wilk normality test (Shapiro and Wilk, 1965) and transformed by Box–Cox transformations (Box and Cox, 1964) using R version 3.2.3 (R Development Core Team, 2015) of the MASS package (Venables and Ripley, 2002) to improve their fit to QTL analysis. The QTL analysis was performed for each environment separately and also using mean data across environments of the trait. Initially, the composite interval mapping (CIM) (Zeng, 1994) program of WinQTL Cartographer version 2.5 (Wang et al., 2005) was used to detect QTL. Quantitative trait loci were scanned across the sunflower genome using forward and backward regression method with the standard model (Model 6) for up to five control markers. A window size of 10 cM was used with a walk speed of 1 cM. The significance thresholds of the LOD ratio values corresponding to P ≤ 0.05 were determined for each environment independently using the genome-wide permutation tests with 1000 replications (Churchill and Doerge, 1994). Only QTL at or above the LOD threshold for a given environment were reported as significant. A 95% confidence interval was used to estimate the left and right margins of the QTL using 1-LOD and 2-LOD of the most likely QTL peak position. The neighboring QTL were required to be at least 10 cM apart at their peaks and not sharing any overlapping genomic regions of 1-LOD support interval. The CIM feature of the QGene version 4.3 software (Joehanes and Nelson, 2008) was used to verify the output of WinQTL Cartographer. The forward-cofactor selection option was applied for automatic selection of cofactors. Significance thresholds for the LOD scores were determined for each evaluation through resampling with 1000 iterations. Finally, the QTL analysis results were confirmed with PLABQTL version 1.2 (Utz and Melchinger, 2006). In contrast to the maximum likelihood approach of WinQTL Cartographer and QGene software, PLABQTL uses a multiple regression procedure described by Haley and Knott (1992). The CIM analysis in PLABQTL was performed using the cov SEL statement for automatic cofactor selection and permute option to derive LOD threshold at the 0.05 significance level from 1000 permutations. Akaike’s information criterion was used for the selection of the best CIM model. We compared the CIM analyses output of the three programs and reported only those QTL in this study that were detected with significant LOD values in at least two programs within the same genomic regions. For a given locus, a positive additive effect indicates that the RHA 439 parent allele conferred resistance against BSR. The linkage map and QTL positions were drawn using the software MapChart 2.2 (Voorrips, 2002). The effect of significant QTL on BSR resistance was performed using PROC ANOM in SAS 9.3 (SAS Institute, 2011).

Marker Enrichment at Quantitative Trait Loci Region

A total of 18 SNP markers were selected from the published sunflower SNP maps (Bowers et al., 2012; Talukder et al., 2014a; Hulke et al., 2015) by comparing a SNP marker position of our map located near a QTL peak position on LG17 (Supplemental Table S2). These 18 SNPs together covered a total of 10.32 Mb physical region on the sunflower genome encompassing the QTL position. Additionally, six public simple sequence repeat (SSR) markers, ORS169, 0RS297, ORS380, ORS735, ORS807, and ORS988, mapped on LGs 10 and 17 of previously reported BSR QTL studies (Davar et al., 2010; Amouzadeh et al., 2013) were surveyed for polymorphism in our population.

Conversion of Single Nucleotide Polymorphisms into Length Polymorphic Markers

Genotyping of the 18 selected SNPs and SNPs flanking the significant BSR resistance QTL regions was performed using a strategy of converting the SNPs into length polymorphism markers. For each SNP, three primers were designed: two forward primers (F1 and F2) and a common reverse primer (Table 1). The forward primers are specific to the two alternate alleles of the SNP and are attached with a tail, 5′-GCAACAGGAACCAGCTATGAC-3′ at 5′ terminus. An additional 5-base oligonucleotide (5′-ATGAC-3′) was inserted between the tail and the sequences of the F2 forward primer to produce length polymorphism between two SNP alleles during amplification. One artificial mismatch was introduced in each forward primer at alternate third and fourth base positions from the SNP allele. The mismatch was done using the following rule: A and T was mismatched to C, G was mismatched to A, and C was mismatched to T. Finally, universal priming-element-adjustable primer, a fluorescence tag attached to the tail sequence at the 5′ terminus (5′-ATAGCTGG-Sp9-GCAACAGGAACCAGCTATGAC-3′), was used in each PCR. The primers were screened for amplification and length polymorphisms among parents and RILs using the methods described by Qi et al. (2015).

View Full Table | Close Full ViewTable 1.

Allele-specific polymerase chain reaction primers for single nucleotide polymorphism (SNP) markers tightly flanked the significant sunflower Sclerotinia basal stalk rot resistance quantitative trait loci (QTL) on linkage groups 10 and 17.

QTL Flanking SNP markers Primer name Primer sequence† Marker property
The tail sequence is in bold and the additional five-base oligonucleotide insertion in AS-primer F2 is bold and underlined; the italic indicates SNP and strikethrough is the introduced mismatch.


Basal Stalk Rot Disease Assessment of Sunflower Recombinant Inbred Line Population

Typical BSR symptoms were observed in all five environments (locations and years). The highest BSR occurrence was observed in the Crookston 2013 environment with the mean DI of 48.3% followed by Carrington 2012 (30.4%), Grandin 2014 (24.2%), and Crookston 2012 (23.7%). The lowest BSR occurrence was observed at Carrington 2014 with mean DI of 10.2% (Supplemental Fig. S1). Across all environments, the mean DI for the parental lines was 29.1 and 18.7% for the HA 441 and RHA 439, respectively, suggesting that both parents were tolerant to BSR (data not shown). The parental lines showed differential BSR response at all environments (Fig. 1). The RHA 439 parent generally showed more tolerance to BSR than HA 441 in every environment except in Crookston 2013. Wide distributions of DI scores were observed among the RILs in all five environments (Fig. 1). Shapiro–Wilk normality test (Shapiro and Wilk, 1965) revealed that the data are not normally distributed. Except for Crookston 2013, the distributions of DI data of the remaining four environments were largely skewed toward the lower DI values. The Pearson’s coefficient of skewness for the Carrington 2012 and 2014, Crookston 2012 and 2013, and Grandin 2014 environments were 0.86, 1.47, 0.81, 0.10, and 0.89, respectively. The Carrington 2014 environment had a DI range from 0 to 47.6%, the lowest among all the environments. Although, the mean DI of the Carrington 2012 environment was moderate, the range of the DI was from 0 to 100% among the RILs. The range of the DI for the Crookston 2012 and Grandin 2014 environments was 0 to 73.2 and 0 to 88.1%, respectively. The susceptible checks, HA 89 and Cargill 270, showed the highest BSR occurrence in the Crookston 2013 environment with a DI of 51.6 and 72.6%, respectively (Supplemental Fig. S1). They also had the lowest DI scores of 22.3 and 24.6%, respectively, in the Crookston 2012 environment. Across the five environments, the resistant check Croplan 305 had the lowest DI in the Carrington 2014 environment (7.9%), while it was the highest in the Crookston 2013 environment (34.9%).

Fig. 1.
Fig. 1.

Frequency distribution of Sclerotinia basal stalk rot (BSR) disease incidence (DI) among 106 sunflower recombinant inbred lines (RILs) screened in multienvironments during 2012 to 2014. The arrowheads indicate the DI levels of the parental lines, HA 441 (green) and RHA 439 (blue). The Shapiro–Wilk normality test statistic (W), the probability value (P), and the Pearson’s coefficient of skewness (Sk) of the data for each environments are shown inside the plots.


The BSR phenotypes were analyzed using ANOVA (Table 2). Highly significant genetic variation (P < 0.001) was observed for the DI in the RIL population. The environments, replication within environment, as well as the genotype × environment interactions were also significant, suggesting their contributions to phenotypic variation of DI across environments. The broad-sense heritability (H2) estimate for BSR on an entry mean basis (0.69) was moderately high (data not shown). Spearman rank correlation coefficients of DI for RILs among all five environments were highly significant (Table 3). However, the magnitude of the correlation coefficient was the highest between the Crookston 2013 and Grandin 2014 environments (ρ = 0.66), while it was the lowest between the Crookston and Carrington 2012 environments (ρ = 0.29).

View Full Table | Close Full ViewTable 2.

Analysis of variance (ANOVA) for Sclerotinia basal stalk rot (BSR) disease incidence (DI) scores among sunflower parents and 106 recombinant inbred lines (RILs) derived from the cross HA 44 × RHA 439 screened in multienvironments during 2012 through 2014.

Component df Variance estimate Confidence limit (0.05)
F/Z value† p-value > F/Z
Lower Upper
Environment 4 164.7 <0.0001
Replication (environment) 9 σ2r = 13.19 5.42 66.09 1.71 0.0439
Genotype 107 σ2g = 161.81 119.21 232.30 5.92 <0.0001
Genotype × environment 428 σ2gl = 61.15 39.35 107.90 3.95 <0.0001
Error 963 σ2e = 345.73 315.86 380.07
In the PROC MIXED model, environments were considered fixed and, therefore, subject to F-test (values in bold). F, Fisher’s F-test statistic; Z, Z-test statistic.

View Full Table | Close Full ViewTable 3.

Spearman rank correlations (ρ) between Sclerotinia basal stalk rot (BSR) disease incidence (DI) scores of 106 sunflower recombinant inbred lines screened at five field trials during 2012–2014 growing seasons.

Environment Crookston 2012 Crookston 2013 Carrington 2014 Grandin 2014
Carrington 2012 0.29** 0.55*** 0.53*** 0.51***
Crookston 2012 0.46*** 0.34** 0.48***
Crookston 2013 0.46*** 0.66***
Carrington 2014 0.57***
**Significant at the 0.01 probability level.
***Significant at the 0.001 probability level.

Genetic Map Construction

Linkage analysis was performed using 1521 SNP markers genotyped in the 106 RILs of the HA 441 × RHA 439 mapping population. A total of 430 markers (28.3%) showed highly significant segregation distortion from the expected 1:1 ratio by the Chi-square test and were excluded from map construction. The remaining SNP markers were placed onto 17 sunflower LGs, except for 42 markers, which could not be suitably placed on LGs. Although the majority of the SNPs substantiate the draft sunflower genome assembly and mapped on the respective LGs, 9.3% (98 of 1049) of the SNPs did not agree with the physical map. Among the selected 18 SNP markers from published sunflower SNP maps, four were found to be polymorphic in our RIL population and mapped on LG17. Out of the six public SSR markers tested, two were polymorphic and mapped one of each on LGs 10 and 17. The final linkage map, therefore, consisted of 1053 SNPs and two SSR markers. The total length of the genetic map was 1401.36 cM with an average distance between markers of 1.33 cM across the 17 LGs. Individual LGs ranged from 30.34 cM in LG7 to 138.03 cM in LG16; the number of markers per LG varied from 12 in LG13 to 174 in LG10. The linkage map was used for QTL analysis. A detailed description of the 17 LGs with marker names and cM positions is shown in Supplemental Table S3.

Quantitative Trait Loci Identification for Basal Stalk Rot Resistance

Composite interval mapping analyses detected significant QTL at each environment for the BSR trait (Table 4). The highest numbers of QTL were identified at the Crookston location in both the 2012 and 2013 environments with three QTL each. In the Crookston 2012 environment, the three QTL were identified on LGs 10, 16, and 17. The phenotypic variation for BSR resistance explained by individual QTL in the RIL population ranged from 8.4 to 16.8%. Among the three QTL detected at the Crookston 2013 environment, one of each were mapped on LGs 10, 11, and 17. The phenotypic variation explained by individual QTL ranged from 8.6 to 18.8%. The Carrington location identified two QTL each in both the 2012 and 2014 environments. In 2012, the QTL were detected on LGs 4 and 10, explaining 6.4 and 23.0%, respectively, of the phenotypic variation for the BSR resistance. However, the two QTL detected at the Carrington 2014 environment were on LGs 9 and 10, each explaining 9.3 and 22.0%, respectively, of the phenotypic variation. The Grandin 2014 environment also detected two QTL, one of each on LG10 and LG17, explaining 28.9 and 26.4%, respectively, of the phenotypic variation for the BSR resistance. The QTL on LG10 was detected at every environment between 57.9- and 66.5-cM genomic positions with high LOD values ranging from 5.5 to 12.0 and was considered a major QTL. The other QTL on LG17 was detected in all but the two Carrington environments, with significant LOD values within a narrow genomic region and, therefore, considered as a major QTL. The resistance allele of the QTL on LGs 4, 9, and 17 were contributed by the HA 441 parent, while the resistance alleles of the QTL on LGs 10, 11, and 16 were contributed by the RHA 439 parent (Table 4).

View Full Table | Close Full ViewTable 4.

Quantitative trait loci (QTL) for Sclerotinia basal stalk rot (BSR) resistance identified in the HA 441 × RHA 439 recombinant inbred line sunflower population in five individual environments.

Environment Linkage group Peak QTL position Flanking markers† (cM position)
LOD‡ Additive effect§ R2 1-LOD interval
Left Right
Carrington 2012 4 32.0 S4_147688288 (29.1) S4_135190076 (33.1) 3.3 −7.2 6.4 27.9–34.4
10 66.5 S10_288646223 (66.5) S10_281294015 (67.5) 8.6 14.4 23.0 63.8–67.3
Crookston 2012 10 60.2 S10_247683641 (60.1) S10_253471019 (60.7) 5.5 8.8 16.8 57.0–61.2
16 87.3 S16_157591485 (87.3) S16_137964301 (87.9) 3.7 6.8 10.5 82.1–92.5
17 23.9 SFW2170 (23.699) S17_228661362 (23.987) 3.1 −6.2 8.36 22.7–24.8
Crookston 2013 10 60.2 S10_247683641 (60.1) S10_253471019 (60.7) 7.1 11.6 18.8 59.5–61.3
11 83.2 S14_148877201 (83.2) S14_148877253 (83.4) 3.6 7.7 8.6 80.9–84.4
17 23.9 SFW2170 (23.699) S17_228661362 (23.987) 3.1 −8.6 12.1 21.2–27.0
Carrington 2014 9 45.0 S9_153762438 (45.0) S9_158145790 (46.6) 3.2 −3.5 9.3 44.5–46.0
10 65.0 S10_273692455 (65.0) S10_271770211 (65.1) 6.9 4.0 22.0 63.0–65.1
Grandin 2014 10 57.9 S10_150172489 (57.9) S10_248236249 (58.8) 12.0 9.9 28.9 57.0–60.0
17 24.0 S17_228661362 (23.987) NSA_002284(24.383) 7.0 −8.6 26.4 23.6–24.3
SNP markers nearest to the QTL pick position are underlined.
LOD, logarithm of odds.
§Additive effect of the QTL was measured in disease incidence (%) of the BSR. Positive additive effects indicate that the RHA 439 parent contributed to the BSR resistance, while the negative additive effects indicate that the HA 441 parent contributed the resistance.
Percentage of phenotypic variance explained by the QTL in the population.

In the combined analysis using overall mean DI across environments, a QTL was detected on LG10 at the 66.5-cM position with even higher LOD (14.1) and R2 (31.6%) values than detected in any of the five individual environments, suggesting a consensus position of the QTL (Table 5). A second QTL was also detected on LG17 at the 23.9-cM position. This QTL on LG17 accounted for 20.2% of the phenotypic variation for the BSR DI. Since there is no established convention for naming QTL in sunflower, we named the QTL using a prefix Q followed by a three-letter descriptor of the phenotype, the LG number, and a serial number. Consequently, Qbsr-4.1, Qbsr-9.1, Qbsr-11.1, and Qbsr-16.1 represented the environment-specific four QTL for BSR resistance on LGs 4, 9, 11, and 16, respectively, while Qbsr-10.1 and Qbsr-17.1 represented the two QTL detected in multiple environments and in the combined analysis on LGs 10 and 17, respectively. A graphical presentation of the six QTL on the linkage map of HA 441 × RHA 439 RIL population is shown in Fig. 2.

View Full Table | Close Full ViewTable 5.

Quantitative trait loci (QTL) for Sclerotinia basal stalk rot (BSR) resistance identified in combined analysis using integrated disease incidence data collected in HA 441 × RHA 439 recombinant inbred line sunflower population across five environments.

Peak QTL position Flanking markers† (cM position)
LOD‡ Additive
R2 1-LOD interval
Left Right
10 66.5 S10_288646223 (66.5) S10_281294015 (67.5) 14.1 9.5 31.6 65.8–67.4
17 23.9 SFW2170 (23.699) S17_228661362 (23.987) 6.9 −5.2 20.2 22.2–26.0
SNP markers nearest to the QTL pick position are underlined.
LOD, logarithm of odds.
§Additive effect of the QTL was measured in disease incidence (%) of the BSR. Positive additive effects indicate that the RHA 439 parent contributed to the BSR resistance, while the negative additive effects indicate that the HA 441 parent contributed the resistance.
Percentage of phenotypic variance explained by the QTL in the population.
Fig. 2.
Fig. 2.

Genomic location of quantitative trait loci (QTL) associated with sunflower Sclerotinia basal stalk rot (BSR) resistance detected using HA 441 × RHA 439 recombinant inbred lines (RILs) mapping population screened in multienvironments during 2012 to 2014. Rectangles represent the one logarithm of odds (1-LOD) confidence interval of the QTL.


Effect of Major Quantitative Trait Loci on Basal Stalk Rot Resistance

Two significant QTL, Qbsr-10.1, and Qbsr-17.1, which were detected in multiple environments, as well as in the overall mean across environments, were used to study the effect of QTL on BSR resistance in the HA 441 × RHA 439 RIL population (Fig. 3). Four possible categories of the allelic combinations were distributed in the RIL population for these two QTL: (i) resistance alleles from both Qbsr-10.1 and Qbsr-17.1 (Q10/Q17), (ii) resistance allele from Qbsr-10.1 and susceptible allele from qbsr-17.1 (Q10/q17), (iii) susceptible allele from qbsr-10.1 and resistance allele from Qbsr-17.1 (q10/Q17), and (iv) susceptible alleles from both qbsr-10.1 and qbsr-17.1 (q10/q17). An analysis of means revealed that the resistance allele combination from both QTL (Q10/Q17) significantly (P < 0.05) reduced the BSR DI in the RIL population, while the susceptible allele combination (q10/q17) significantly increased the DI in the RIL population in all environments (Fig. 3). A similar response was also observed when integrated mean DI data of all environments was analyzed. In case of mixed allele combinations, for instance, the resistance allele of Qbsr-10.1 and susceptible allele of qbsr-17.1 (Q10/q17) combination generally reduced the DI below the RIL population mean in all but two Crookston environments. On the contrary, the susceptible allele of qbsr-10.1 and resistance allele of Qbsr-17.1 (q10/Q17) combination was less effective in reducing the DI in the RIL population in all environments and also when mean DI data across environments was analyzed. However, the response of the mixed allele combinations (Q10/q17 and q10/Q17) to the BSR DI was not statistically significant in any environment (Fig. 3).

Fig. 3.
Fig. 3.

Analysis of means (ANOM) test shows the effects of allelic combinations of two major quantitative trait loci (QTL) associated with Sclerotinia basal stalk rot (BSR) resistance in sunflower recombinant inbred line (RIL) population derived from the cross, HA 441 × RHA 439 evaluated in five environments during 2012–2014 growing seasons. The first allele is from the QTL Qbsr-10.1 on LG10 contributing resistance from RHA 439 parent, while the second allele is from the QTL Qbsr-17.1 on LG17 contributing resistance from HA 441 parent. A light blue area inside each plot shows the upper and lower bounds of decision limits. Values outside decision limits are significantly different (P < 0.05) from the mean disease incidence (DI) of the RIL population. The dark blue columns represent the mean DI of the RILs possessing different allelic combinations of the two major QTL.


Length Polymorphic Markers Assays

A total of nine SNP markers flanking the QTL Qbsr-10.1 and Qbsr-17.1 were converted into allele-specific polymorphic PCR markers (Table 1). All primer sets produced codominant polymorphic amplicons between the parents of the HA 441 × RHA 439 mapping population. These PCR-based SNP markers segregated in the RIL population, which exactly matches the GBS genotyping data for the respective SNPs (Fig. 4).

Fig. 4.
Fig. 4.

Polymerase chain reaction (PCR) amplification of two tightly linked single nucleotide polymorphism (SNP) markers (S10_288646223 and S17_228661362) associated with Sclerotinia basal stalk rot (BSR) resistance QTL Qbsr-10.1 and Qbsr-17.1 in sunflower. Allele-specific PCR primers designed for these SNP markers produced codominant polymorphic bands for the parents, HA 441 and RHA 439, and showed their segregation in the RIL population. The genotyping-by-sequencing (GBS) genotype data of the respective SNP markers (shown in the bottom rows) matches the band sizes produced by the allele-specific PCR primers.



Breeding for resistance to Sclerotinia has been a major goal of sunflower improvement programs in cool and humid production regions of the world. To gain insight into Sclerotinia resistance against root infection caused by myceliogenically germinated sclerotia, BSR has been studied using a set of RILs generated from a cross between two sunflower lines, HA 441 and RHA 439. The parents of the RIL population showed moderate to high level of tolerance to BSR in the field as well as in the greenhouse trials. Phenotypic evaluation revealed a wide variation of DI from susceptibility toward resistance among RILs in all environments (Fig. 1) consistent with the typical quantitative disease resistance response where resistance is conferred by the action of many genes, each of which has a small effect on the levels of disease observed (Roux et al., 2014). Transgressive segregation was observed for the trait in all five environments in the RIL population, where some of the progeny showed more extreme phenotypes than either of the parents, which is frequently observed in plants (de Vicente and Tanksley, 1993). This suggests that both the parents of the RIL population are contributing to the BSR resistance. Despite our effort to create high disease pressure using artificial inoculation, the mean DI varied greatly among growing seasons as well as between the different field trail locations (Fig. 1). Significant environmental variation could be attributed by environmental factors such as humidity, temperature, and moisture, which are critical for optimum disease development (Gulya et al., 1997). The frequency distribution of DI data exhibited skewed distribution toward the more resistant phenotype. The skewness appeared to be more prominent in environments with lower mean DI, which was evident from the highly significant negative correlation of mean DI with skewness (r = −0.95) (data not shown). Nonconducive environmental conditions for BSR disease development might have benefitted the moderately resistant RILs to escape disease development and skew them toward the more resistant group. Skewed distributions of disease phenotypes in QTL mapping populations have also been observed in other host–pathogen systems (Sharma et. al., 2009; Rant et al., 2013; Berger et al., 2014). Nevertheless, this does not impair our ability to detect QTL, since the data were appropriately transformed to fit a normal distribution, a prerequisite for handling such data in QTL mapping analysis using the maximum likelihood method (Yang et al., 2006).

The phenotypic responses of the RIL population to the BSR disease appeared to be influenced by the environmental conditions as evident from the ANOVA analysis (Table 2). Both genotype and environment, and their interaction, had highly significant effects on the BSR DI. The variance component from genotypes was higher than their respective variance component for the genotype × environment interaction, suggesting that the variation for DI among the RILs was mainly contributed by the genotypes. Broad-sense heritability (H2), which estimates the proportion of phenotypic variance that is due to genetic factors, was moderately high (69%), lending further validation to our assumption that the observed variation for DI was mainly explained by the genetic makeup of the RILs. This result is consistent with our previous observations for this trait, where the genotype had much larger effect than the genotype × environment interaction (Talukder et al., 2014b). Despite significantly different environments and highly variable mean DI across environments, the Spearman’s rank correlation coefficients for DI among environments were highly significant (Table 3), suggesting a general agreement of repeatability of the screening trials across different environments.

Single nucleotide polymorphisms are now the predominant marker system of choice in modern genomics research (Ganal et. al., 2009). Single nucleotide polymorphisms are stable, abundant across the genome, amenable to high-throughput genotyping platforms, and highly cost-effective. A NGS-technology-based GBS approach, which has the advantage of simultaneous discovery and genotyping of a large numbers of SNP markers (Elshire et al., 2011), was used to genotype the RIL population. Moreover, the GBS approach practically eliminates the labor-intensive, time-consuming, and, often, expansive polymorphism screening process associated with other marker systems. To the best of our knowledge, this is the first report of using SNP markers generated through GBS in sunflower to map QTL associated with traits of any kind in a biparental mapping population. Even though we used the highly efficient GBS approach for SNP discovery in the RIL population, we mapped only 1053 SNP markers across 17 sunflower LGs, which is a modest number compared with other studies (Poland et al., 2012; Spindel et al., 2013; Chen et al., 2014). Both the parents of the sunflower RIL population were public USDA–ARS cultivated inbred lines. Cultivated sunflower is known to have a narrow genetic base (Korell et al., 1992) and apparently originated from a small number of ancestral germplasm sources (Cheres and Knapp, 1998). Thus, a low level of polymorphism is not unexpected in an intraspecific sunflower population as observed in other species known to have narrow genetic bases (Jaganathan et al., 2014). The length of the linkage map was 1401.36 cM, which is comparable with the recently developed sunflower consensus linkage map of 1443.84 cM (Talukder et al., 2014a) and also the combined sunflower map of 1310 cM constructed using 10,080 marker loci (Bowers et al., 2012). Despite adequate coverage of the sunflower genome, several regions with gaps >25 cM were observed without markers. Most notable were single regions on LGs 1, 9, 11, 12, 14, 16, and 17 and two regions on LGs 5 and 15 (Supplemental Table S3). A possible reason for the lack of polymorphic markers in these gaps likely is due to the sharing of similar identity-by-descent genomic regions between the mapping parents.

Because of the presence of significant genotype × environment interactions, QTL analyses were first performed for each of the five environments separately followed by a combined QTL analysis using mean BSR DI across environments. The number of detected BSR resistance QTL ranged from two to three QTL per environment (Table 4). The phenotypic effect of each of the QTL explained between 6.4 to 28.9% of the BSR variations in the RIL population. Given the quantitative nature of the BSR resistance (St. Clair, 2010), there appears to be a relative paucity of resistance QTL in each environment. The modest population size used in QTL mapping, like the one used in the current study, is likely to reduce the power of the QTL significance tests resulting in an underestimation of the number of QTL involved in a trait. Simultaneously, the effects of QTL that are detected with small progeny sizes are also overestimated (Beavis, 1994).

The BSR resistance QTL Qbsr-10.1 was detected in all five environments (Table 4). This QTL was mapped within a narrow genomic region on LG10 in each environment with high LOD values (5.5–12.0). Combined analysis across environments also detected the Qbsr-10.1 QTL with a LOD value of 14.1 at the 66.5-cM peak position. The phenotypic effect for the BSR resistance explained by the QTL was 31.6% in the RIL population. Owing to its large effect and consistent detection across environments, this QTL is considered as the most stable BSR resistance major QTL in our study. The other large-effect QTL, Qbsr-17.1, was detected in three out of the five environments with significant LOD values (≥3.0) and accounted for 8.36 to 26.4% of the phenotypic variation for the BSR DI trait. In the combined analysis, this QTL was detected with a high LOD value (6.9) and explained 20.2% of the phenotypic variation. This QTL was not detected in either of the Carrington environments. However, there was a peak observed on the LOD graph at 23.7 cM position with LOD value of 2.74 in the Carrington 2014 environment (data not shown). This peaks in the LOD graph narrowly missed the significant LOD threshold of 3.0 to be declared significant QTL. Nonetheless, this putative QTL presumably contributed to the phenotypic variation for BSR resistance in the Carrington 2014 environment. All other significant QTL were detected in only one environment. However, it does not necessarily mean that there is no effect of these QTL regions in other environments where these QTL were not detected. For example, Qbsr-11.1 on LG11 was detected with significant LOD value (3.6) in the Crookston 2013 environment. However, there were small peaks observed on LOD graph at similar genomic positions (80.7–83.4 cM) with LOD values >1.0 in the Carrington 2012 and Grandin 2014 environments (data not shown). In the combined analysis, a peak of 2.55 LOD score narrowly failed to reach the significant LOD threshold at 83.2 cM position on LG11. Similarly, a peak of 1.15 LOD was observed on the LOD graph in the Grandin 2014 environment at the 50.4-cM position on LG9, where a significant QTL was detected in the Carrington 2014 environment.

Overall, a significant portion of the phenotypic variation still remains undetected in this study. This could be due to a large number of minor-effect QTL that could not be detected with significant LOD thresholds in the presence of random environmental variation. Future studies using larger mapping populations, robust and precise disease phenotype across wide environments, and improved statistical methods need be employed to identify the role of minor QTL in BSR resistance in sunflower.

Genetic research on BSR resistance QTL mapping in sunflower has been limited. The unavailability of a reliable inoculation technique needed for large-scale field screening trials that simulate the natural infection conditions had hindered the BSR breeding progress in sunflower. The development of an artificial inoculation method by Gulya et al. (2008) provided the opportunity to evaluate a large number of sunflower lines for BSR in multiple field environments (Talukder et al., 2014b,c). Prior to our study, there were only two reports of QTL mapping for BSR resistance in a sunflower RIL mapping population, PAC2 × RHA266 (Davar et al., 2010; Amouzadeh et al., 2013). Both studies were conducted under controlled growth chamber conditions but with different fungal isolates. Davar et al. (2010) identified seven QTL for percentage necrotic area on LGs 1, 2, 4, 6, 8, 14, and 17 in the PAC2 × RHA266 RIL population. We also detected BSR resistance QTL on LGs 4 and 17 in our study. However, the genomic position where we mapped the Qbsr-4.1 was on the opposite end of the LG4 where Davar et al. (2010) identified their QTL. Comparing the genomic position of the common SSR marker (ORS297) on LG17, the Qbsr-17.1 in our map was located downstream from the QTL detected by Davar et al. (2010) on LG17 (Fig. 2). Amouzadeh et al. (2013) screened the same RIL population with fungal isolate SSKH41 and identified five QTL on LGs 1, 3, 8, 10, and 17. Among them, LGs 10 and 17 were common with our study, where we also identified BSR resistance QTL. Comparative mapping using common SSR marker (ORS380) suggest that the QTL detected by Amouzadeh et al. (2013) on LG10 roughly coincides with the genomic location where we identified Qbsr-10.1 in our study (Fig. 2). However, the QTL on LG17 does not coincide with the genomic location where we mapped Qbsr-17.1.

Given the large effect and integrated mapping across environments, the QTL on LGs 10 and 17 are favorable targets for MAS in the development of BSR-resistant sunflower hybrids with wide adaptability. The resistance allele of Qbsr-10.1 was contributed by the RHA 439 parent, while the resistance allele of Qbsr-17.1 was contributed by the HA 441 parent. We have tested the potential of these QTL for MAS breeding using an analysis of means study. The resistance allele combination from both the QTL parents (Q10/Q17) significantly reduced the BSR DI compared with the RIL population mean in all environments and the integrated data (Fig. 3). In contrast, the susceptible QTL allele combination from both the parents (q10/q17) was associated with significantly higher BSR DI in all environments. Interestingly, when the resistance allele was contributed alone by the RHA 439 parent (Q10/q17), DI was reduced below the population mean in almost every environment. However, it was not observed when resistance alleles were only contributed by the HA 441 parent (q10/Q17). This suggests that to gain the maximum benefit from the MAS, the introgression of both resistance alleles of the Qbsr-10.1 and Qbsr-17.1 should be considered. If that is not possible, the second choice would be the resistance alleles of Qbsr-10.1 that would considerably increase the BSR resistance in the introgression lines. To facilitate MAS breeding program, flanking SNP markers encompassing these two QTL were converted into PCR-based allele-specific length polymorphic markers (Table 1). These markers clearly distinguish the parents and the RILs for the resistance–susceptible alleles (Fig. 4), which would be of interest to breeders in MAS breeding programs to introgress the BSR resistance into elite sunflower lines.

In summary, we applied the GBS approach for SNP discovery and construction of the linkage map of the HA 441 × RHA 439 sunflower RIL population. The map contained 1053 SNP markers in 17 LGs spanning 1401.36 cM. Also, this study identified two BSR resistance QTL on the genetic map of LGs 10 and 17. In addition, the inexpensive and technically less demanding allele-specific markers developed in this study can be used immediately in a MAS breeding program with a modest laboratory setup by the small sunflower companies interested in introgression of the BSR-resistant QTL into elite hybrids. Given the quantitative nature of the trait, a larger mapping population involving diverse genetic backgrounds with high level of resistance could provide a broad spectrum of QTL conferring BSR resistance. Studies designed to test this hypothesis are currently underway with a mapping population developed using wild sunflower species.

Supplemental Information Available

Supplemental Figure S1. The panel shows the disease incidence (DI) of the parents, two susceptible checks, HA 89 and Cargill 270, one resistant check Croplan 305, and the 106 RILs of the HA 441/RHA 439 mapping population across five environments evaluated for Sclerotinia basal stalk rot (BSR) during 2012–2014 growing seasons.

Supplemental Table S1. Sequences of the 1236 SNP markers developed by genotyping-by-sequencing (GBS) protocol used for linkage mapping of HA 441/RHA 439 sunflower RIL mapping population.

Supplemental Table S2. Sequences and map positions of the 18 SNP markers on LG17 selected from published sunflower SNP maps.

Supplemental Table S3. Marker ID and cM positions of 1053 SNP and two SSR markers mapped in the 17 linkage groups of sunflower using HA 441/RHA 439 RIL mapping population.


The authors would like to thank Dr. Loren Rieseberg for providing access to the Sunflower Genome Data Repository. We acknowledge the collaboration of Dr. Kevin McPhee and our retired colleague Dr. Tom Gulya during the early part of the project. We thank Drs. William Underwood and Shaobin Zhong for critically reviewing the manuscript. We are grateful to Dr. Yunming Long for his help in designing the allele-specific PCR primers and assay protocol. We also thank Angelia Hogness for her assistance in the lab and Chris Misar, Michelle Gilley, and Megan Ramsett for their assistance in the field. This research was supported by the USDA–ARS National Sclerotinia Initiative, grant number 5442-21220-028-00D and the USDA–ARS CRIS Project No. 5442-21000-039-00D. Mention of trade names or commercial products in this report is solely for the purpose of providing specific information and does not imply recommendations or endorsement by the USDA. The USDA is an equal opportunity provider and employer.




Be the first to comment.

Please log in to post a comment.
*Society members, certified professionals, and authors are permitted to comment.