Challenges of Using Artificial Intelligence in the Process of Shi’i Ijtihad

Round 1

Reviewer 1 Report

·      Thesis

o   From the abstract: current problems in generative AI, from bias to hallucination, make it unusable for Islamic interpretation, but someday perhaps it could be

o   Given the importance of AI in technical spheres and ijtihad in Islamic spheres, the overall topic is quite important

o   But the thesis is not exactly profound, and the optimism for future generative AI is perhaps misplaced. I don’t know that it takes 20 pages to tell me that an AI system that invents facts is not yet suited to making significant religious decisions for human beings. I’d like to see a more compelling thesis emerge around one of the key issues for how it might become functional or relevant, or whether or not it’s artificial status makes it inherently unacceptable regardless of reliability.

·      Logic and argumentation

o   Argues that different problems appear when the model operates as a jurist from when it operates to assist a jurist – it seems like this could have been the place for a very specific kind of thesis

o   Confuses narrow and weak AI – narrow AI is AI of a limited scope, weak AI is AI that doesn’t not “understand” (compared to strong AI, which would understand its operations and perhaps be conscious); AGI is AI that can do anything a human can; but that doesn’t necessarily mean it can understand its output. You can take a weak or strong stance on both narrow AI and AGI.

o   Confuses the matter of hallucination. The point of hallucination is not that you end up with “surreal” results, but rather simply that they aren’t grounded in reality. In fact, when the hallucination is surreal it is easily identified as such. The problem is that the system can invent completely believable but factually incorrect statements. The author’s examples of generating book summaries and invented proverbs are good ones; but the author’s description of the problem is incorrect.

o   The differential between Sunni and Shi’a source material available for LLM training is quite interesting, and speaks to broader issues in religious data for LLMs – again, it feels like there could have been something more specific about how to build an LLM for Shi’a use

o   The author asks the question of whether an AI could be given authority to issue fatwas. I think a more interesting question will be whether jurists will be required to develop facility with AI as one of the disciplines where they must prove their ability. Even sticking with the original question, the author could have tried to articulate under what conditions that authority could be granted.

·      Writing

o   Overall, the writing is clear and workable

o   Line 198: ‘from’ not ‘for’?

·      Source material

o   The paper is quite weak from the perspective of academic engagement. Most of the sources are youtube videos and online articles with little attention to what has been argued in the academic arena

o   For example: there has been lots of work on religion and digital technologies, including (for example) the work of people like H Campbell and T Hutchings on questions of authority and community in online religion

o   For example in AI: there is plenty of work on things like bias mitigation (people like J Buolamwini and T Gebru) or trustworthy AI

o   Whether this is a problem is up to the editors to decide

·      Overall

o   An interesting repercussion of the complexity of ijtihad is that the analysis raises the possibility that a great deal of other human thought is simply way more complex than AI advocates want to believe (especially for AGI).

o   This is a solidly descriptive essay, but I’m not seeing a strong or creative thesis. Much of the explanation belongs in an essay to help religious practitioners think about AI but may be pitched incorrectly for an academic audience.  I think the author should condense some of the explanatory material and delve more into the particulars that ijtihad raises for the future of AI and society.

Author Response

Dear Reviewer, 

Thank you very much for taking the time to carefully read and comment on my article. I sincerely appreciate your valuable insights. Please find my responses to your comments in the attached file.

Best regards,

Author

Author Response File: Author Response.pdf

Reviewer 2 Report

The topic is developing and of contemporary interest. It is refreshing to have studies on artificial intelligence in a variety of world religions.

That said, in my view, the article is currently too general, and should be developed more to focus more specifically on Shi'i ijtihad, before it is ready to be released. It would also benefit from more focus and specific citation of Shi'i scholars or thinkers. The following are some suggestions.

1. Currently, the article intermixes (a) ijtihad (in the sense of assisting highly trained Shi'i scholars in producing fatwas), (b) providing existing fatwas to laity (like Google, apps, and bots already do), (c) AI independently generating new fatwas to laity without the intervention of Shi'i scholars, and (d) counselling/pastoral care. 

These should each be treated separately, and the article should specify which is being discussed.

For instance, in actual ijitihad (as a scholarly tool), privacy is not a consideration. Accessibility is also unlikely to be a consideration (most high-level scholars would be using purpose-built software, not software from OpenAI, and don't work from remote villages or battlefields).

Basically, the concerns over AI that are stated are general issues regarding the use of ChatGPT globally, not things specific to ijtihad. So it would be make the article stronger if it were more specific.

Similarly, some of the references are more general and not Islam-focused; it would be stronger if the references were more Islam-focused. 

2. The article would be stronger if it considered varying approaches to ijtihad, including both traditional views but also "reformist" views. Algorithms do well at "traditional" approaches (literal readings) but not as well at things like social contextualization or considering the ethos of the shariah. Maybe the views of Haider Hobollah might be relevant. There is always Abdolkarim Soroush, who is also in ChatGPT's dataset.  

3. The article would be strengthened by directly mentioning views of high-level Shi'i scholars on AI, or more ways that it is currently being used. 

4. It would be good to rewrite the article to be more focused on the specific topic; some of it (like Buddhist robots) is relevant to AI and religion in general, but it ends up being too broad. 

5. In my experience, ChatGPT has these specific additional problems when discussing Islam:

* It can't "conceptualize" historical time periods and understand that some ideas change over time or are relevant to some time periods and not others. For instance, abrogation in the Qur'an. 
* It has difficulty disambiguating medieval Muslims who have multiple or similar names, e.g. figuring out when "Abu Ali", "Ibn Sina", "Bu Sina", "Abu Ali Sina", and "Avicenna" are the same person. This would be a nightmare for studying hadith narrators. 
* Probably it has others; it would be a good contribution to identify more. 

6. Rather than discussing general issues surrounding ChatGPT/digital technologies, the article would be stronger if it engaged in (or cited) solid case studies using this technology to explore things like fatwas or Qur'anic exegesis. Currently the references are general and not specific to Islamic studies.

7. Here are some suggested writings that the author could consider reading and integrating into the piece:

Boumediene Hamzi, ‘Initial Insights on the interface of Mathematics, Machine Learning, and Islamic Philosophy’ [unpublished conference paper,  available from the author]

Abid et al., ‘Large language models associate Muslims with violence’, in arXiv <https://arxiv.org/abs/2101.05783>.

‘Debiased Large Language Models Still Associate Muslims with Uniquely Violent Acts’, in arXiv https://arxiv.org/abs/2208.04417

Patel et al., ‘Building Domain-Specific LLMs Faithful to The Islamic Worldview: Mirage or Technical Possibility?’, in arXiv <https://arxiv.org/abs/2312.06652>

Ali-Reza Bhojani and Marcus Schwarting, ‘Truth and Regret: Large Language Models, the Quran, and Misinformation’ - On the problem of false information when using ChatGPT to discuss Islam (including problems with rendering Qur’anic verses correctly, which is a basic for Islamic scholarship)

Amina Inloes, “Robo-shaykhs: Can artificial intelligence derive fatwas? A Twelver Shiʿi view” (conference paper; forthcoming 2024 in Al-Qalam journal) 

Mohammad Atari, Mona J. Xue, Peter S. Park, Damián Blasi, and Joseph Henrich, ‘Which Humans?’ [on AI cultural bias, including compared to Muslim-majority cultures]


Amina Inloes, ‘Artificial Intelligence and Islam: Who’s (Not) Afraid of ChatGPT?’, in Islamic Perspectives on the Beginning of Human Life: Jurisprudential, Theological and Social Implications (conference paper; Birmingham: AMI Press, forthcoming 2024)

 Amina Inloes, ‘Artificial Intelligence and Islamic Theology: An Interview with ChatGPT’ (conference paper from the Eighth Annual Conference on Shi'i Studies, expected publication April 2024)

Emad Hamdeh, Shaykh Google as Ḥāfiẓ al-ʿAṣr, “The Internet, Traditional ʿUlamā’, and Self Learning” - Some of the issues regarding traditional community and humans are brought up here

Drefyus, On the Internet, also brings up the question of the internet and transmission of knowledge 

At the BRAIS (British Association for Islamic Studies) conference in 2023, an introductory talk was about AI and manuscript research, and that could be worth digging up as an alternative to LLM's

Additionally, there is a research group for artificial intelligence and Islam (based out of London but accessible worldwide virtually) that the author may be interested in participating in if they are not already. For more information, they can visit: https://sites.google.com/view/islam-ai-rg

8. References: Be consistent in using Farsi script or transliteration

-- Specific line-by-line comments -- 

Lines 16-21: I can understand putting a paragraph about hijab at the beginning to whet the reader's interest.

However, firstly, I find it uncomfortable - the question of whether or not shariah says that women should wear hijab is different from the question of whether women should be killed for not doing it/whether governments should enforce it. Most Muslim-majority countries do not enforce hijab, and there is nothing in the Quran or Shii hadith saying that governments should punish women for their clothing. Rather, this was a decision made after the Revolution. I think it is better to treat human rights issues as essential freedoms (freedom to practice or not practice one's faith in one's own way) not as shariah questions.

Second, a LLM would likely do very poorly at coming up with a ruling for hijab. First, this is because it has a tremendous dataset about debates over hijab among contemporary Muslims, and it's too much. Second, the way that hijab is discussed in Shi'i hadith and exegesis is somewhat circuitous and reliant on ijma'. Third, there are other issues such as social custom or changes in custom over time, that an LLM would likely handle poorly. A more boring example, like dietary laws, would probably be done better by an LLM. 

But, again, this is a good place for a case study - how well does it do with an in-depth discussion? 

27 – “Finding an individual possessing expertise across these diverse fields is challenging” - I disagree; in my experience, genuine mujtahids who have studied for decades do tend to have encyclopaedic knowledge of the sources, and this is bolstered by "instant-mujtahid" software such as hadith software. Basically, I'm not sure that AI would really uncover anything that humans don't already have. But I could be proven wrong; this would be an excellent place for a case study.

37 – “This response highlights the impressive breadth of knowledge possessed by this artificial intelligence model” – I would disagree with that. For one thing, the actual arguments for hijab are heavily hadith-based, not based on exegesis of the Qur'an. Many contemporary discussions about hijab also focus on questions such as social contextualization, what about the laws saying slave women should not wear hijab, etc, that don’t tie directly into the Qur’anic verses.

61-110 - Section 1.1 – Possibly, this section could be abridged (at the editor’s discretion) since basic knowledge about AI is more common these days. I do understand that information about AI that was new and exciting last year is humdrum by now due to the vast amount of information put out by the media.

179 – Changes in rulings are rarely due to sources being overlooked and later discovered. Rather, they are often spurred by changing social values or circumstances, which then prompt scholars to read texts differently or challenge the historically common views. For instance, the questions of the acceptability of music or the ritual purity of Ahl al-Kitab were both impacted by modernity and contact with the West and Western technologies. One could also consider the impact of changing views on hadith narrators, or how strict to be about the authentication of hadith.

(If the author disagrees, their argument for their stance could be clarified)

181 – If an AI is trained in the traditional process of ijtihad, it seems unlikely that it would really result in reforming religious rulings. (Or, if it did, it would be quickly shut down.) Rather, it would probably just replicate what traditional scholars say. It would probably only result in new rulings if it were actually given a new methodology, or allowed to develop its own methodology.

 120-122 I don't think it adds to the paper for the author to comment on his/her academic pursuits; we expect the author is a specialist in Shi'ism. 

280 – The question of in-person religious communities may not be relevant here, since most Shi’a do not have direct contact with mujtahids and, today, often take fatwas from books, apps, television, or the internet

325 – “The primary bias is the distortion of the interpretation of sacred texts” , this would be a good point to explore and develop, perhaps with citing case studies of interpretations of specific verses of the Qur’an. (The reference cited, Coded Bias, seems unlikely to be deep-diving into Qur’anic exegesis.) How might ChatGPT be distorting the Qur'an, specifically?

37- - Exclusion of marginalized opinions – In my experience, when discussing Islam, ChatGPT does in fact prioritize non-traditional/“reformist” views over more “orthodox” ones. I suspect this is because it has a stronger English-language dataset, and non-traditional views get more airtime in English than traditional views. In any case, this is just my experience; it would be helpful if the article explored this more rigorously with examples.

488 “Who did he love the most?” – It is somewhat paradoxical to ask a question about love to a being which is incapable of love, and it also brings up the question if love can really be quantified, since we often love different people in different ways (even though, theoretically, a machine would try to quantify it). I understand it is treated in hadith and is a sectarian issue, just observing.

Anyway, HadithGPT was known to be very limited due to its small dataset (only specific, limited hadith books), and so it is not surprising that it gave inconsistent responses.

562 – “There is a growing body of scholarly works that argue for AI’s role solely as an assistant in religious matters” – The reference is not Islam-focused; it would be helpful to cite something specifically about Islamic scholarship. 

586 – I guess one way to decide whether or not an AI could get a license for ijtihad would be a sort of Turing test – if it demonstrates, in practice, the ability to do it which is equal to or better than that of human mujtahids, then why not. That said, this would be a good place to explore traditional criteria for mujtahids and to see whether or not they would apply to a machine (and, by extension, whether some of these criteria should be challenged).

643 – It was worthwhile to mention the statistics about the popular acceptance of computer-derived fatwas – in the era of the democratization of knowledge, the issue of popular acceptance among the laity is important. (Not just the question of acceptance among scholars). In the end, it is likely the laypeople, rather than the high-level scholars, who will decide whether or not this technology is a legitimate authority through their actual use of it, similar to how people use Google to find fatwas today.

697 – I haven’t come across any Muslim scholars saying that using AI would lead to wahn or weakening Islam. Rather, there seems to be the reverse – the view that Islam promotes science and technology, and Muslims are keeping up to date and showing we are modern and relevant by using new technology. This could be explored more.

723 - Unexplainability - Well, no one really knows how humans think either; the key point would be it would need to explain its evidence for its decision. Also, some maraji’, like Ayatollah Sistani, do not publicize their procedure in deriving specific fatwas to the masses.

845 – Whether or not the process of ijtihad can be dehumanized is an essential question. That is, is it solely procedural, or does it require something uniquely human? Also, some scholars will bring up the question of lotf (divine grace) in deriving fatwas; can AI experience these spiritual phenomena? (It’s a question that, in my view, doesn’t have a clear answer, but is worth exploring.)

---

In short, I understand that is a heavy dose of "constructive criticism", but I think that taking a more focused approach on questions of Shi'i scholarship would genuinely result in a new and exciting paper, rather than one that is more generic. But congratulations to the author on getting this far, and treating an important current topic! 

Author Response

Dear Reviewer, 

Thank you very much for taking the time to carefully read and comment on my article. I sincerely appreciate your valuable insights. Please find my responses to your comments in the attached file.

Best regards,

Author

Author Response File: Author Response.pdf

Reviewer 3 Report

Dear Author, I have reviewed your paper and found it to be interesting. However, I have a few comments and suggestions for improvement: - Language of References: Some references in the paper are written in Persian, which may limit accessibility for readers who do not understand the language. It would be beneficial to translate these references into English. - Definition of Study Terms: It would be beneficial to to include them under another title after the introduction where you define the key terms used in your study. - I noticed that in line 88, the word "automatically" appears to be underlined. - I would also suggest that you consider incorporating more references to strengthen the credibility and depth of your study.

Author Response

Dear Reviewer, 

Thank you very much for taking the time to carefully read and comment on my article. I sincerely appreciate your valuable insights. Please find my responses to your comments in the attached file.

Best regards,

Author

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Since very few changes were made to the paper, I would like to recommend that the author re-read my recommendations and give them more consideration. 

After reading the author's responses, I would like to add the recommendation that if the author is saying something based on a specific Shi'i scholar (e.g. "I refer you to Ayatollah Mubalighi's lecture on the subject, which is cited in the paper"), it would be helpful to present what is being said as that scholar's viewpoint, by naming that specific scholar in the paper and quoting what they are saying, rather than just having footnotes, without it being apparent who the idea is coming from. This would provide a more specific level of discussion. 

Author Response

Dear Reviewer,

I sincerely appreciate your time and effort in reviewing my responses to your previous comments. Based on your feedback, I have made the necessary revisions to align with your expectations in the second round of comments.

I would also like to extend my apologies if any of my previous responses seemed inappropriate or came across as overly direct. Please know that it was not my intention to be disrespectful.

Thank you once again for your patience and guidance.

Warm regards,
Hasan Latifi